Mercurial > hg > graal-jvmci-8
annotate src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.cpp @ 1716:be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
Summary: GC workers now recognize an intermediate transient state of blocks which are allocated but have not yet completed initialization. blk_start() calls do not attempt to determine the size of a block in the transient state, rather waiting for the block to become initialized so that it is safe to query its size. Audited and ensured the order of initialization of object fields (klass, free bit and size) to respect block state transition protocol. Also included some new assertion checking code enabled in debug mode.
Reviewed-by: chrisphi, johnc, poonam
| author | ysr |
|---|---|
| date | Mon, 16 Aug 2010 15:58:42 -0700 |
| parents | 63f4675ac87d |
| children | 5ed703250bff |
| rev | line source |
|---|---|
| 0 | 1 /* |
|
1552
c18cbe5936b8
6941466: Oracle rebranding changes for Hotspot repositories
trims
parents:
1521
diff
changeset
|
2 * Copyright (c) 2001, 2010, Oracle and/or its affiliates. All rights reserved. |
| 0 | 3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. |
| 4 * | |
| 5 * This code is free software; you can redistribute it and/or modify it | |
| 6 * under the terms of the GNU General Public License version 2 only, as | |
| 7 * published by the Free Software Foundation. | |
| 8 * | |
| 9 * This code is distributed in the hope that it will be useful, but WITHOUT | |
| 10 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or | |
| 11 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License | |
| 12 * version 2 for more details (a copy is included in the LICENSE file that | |
| 13 * accompanied this code). | |
| 14 * | |
| 15 * You should have received a copy of the GNU General Public License version | |
| 16 * 2 along with this work; if not, write to the Free Software Foundation, | |
| 17 * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. | |
| 18 * | |
|
1552
c18cbe5936b8
6941466: Oracle rebranding changes for Hotspot repositories
trims
parents:
1521
diff
changeset
|
19 * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA |
|
c18cbe5936b8
6941466: Oracle rebranding changes for Hotspot repositories
trims
parents:
1521
diff
changeset
|
20 * or visit www.oracle.com if you need additional information or have any |
|
c18cbe5936b8
6941466: Oracle rebranding changes for Hotspot repositories
trims
parents:
1521
diff
changeset
|
21 * questions. |
| 0 | 22 * |
| 23 */ | |
| 24 | |
| 25 # include "incls/_precompiled.incl" | |
| 26 # include "incls/_concurrentMarkSweepGeneration.cpp.incl" | |
| 27 | |
| 28 // statics | |
| 29 CMSCollector* ConcurrentMarkSweepGeneration::_collector = NULL; | |
| 30 bool CMSCollector::_full_gc_requested = false; | |
| 31 | |
| 32 ////////////////////////////////////////////////////////////////// | |
| 33 // In support of CMS/VM thread synchronization | |
| 34 ////////////////////////////////////////////////////////////////// | |
| 35 // We split use of the CGC_lock into 2 "levels". | |
| 36 // The low-level locking is of the usual CGC_lock monitor. We introduce | |
| 37 // a higher level "token" (hereafter "CMS token") built on top of the | |
| 38 // low level monitor (hereafter "CGC lock"). | |
| 39 // The token-passing protocol gives priority to the VM thread. The | |
| 40 // CMS-lock doesn't provide any fairness guarantees, but clients | |
| 41 // should ensure that it is only held for very short, bounded | |
| 42 // durations. | |
| 43 // | |
| 44 // When either of the CMS thread or the VM thread is involved in | |
| 45 // collection operations during which it does not want the other | |
| 46 // thread to interfere, it obtains the CMS token. | |
| 47 // | |
| 48 // If either thread tries to get the token while the other has | |
| 49 // it, that thread waits. However, if the VM thread and CMS thread | |
| 50 // both want the token, then the VM thread gets priority while the | |
| 51 // CMS thread waits. This ensures, for instance, that the "concurrent" | |
| 52 // phases of the CMS thread's work do not block out the VM thread | |
| 53 // for long periods of time as the CMS thread continues to hog | |
| 54 // the token. (See bug 4616232). | |
| 55 // | |
| 56 // The baton-passing functions are, however, controlled by the | |
| 57 // flags _foregroundGCShouldWait and _foregroundGCIsActive, | |
| 58 // and here the low-level CMS lock, not the high level token, | |
| 59 // ensures mutual exclusion. | |
| 60 // | |
| 61 // Two important conditions that we have to satisfy: | |
| 62 // 1. if a thread does a low-level wait on the CMS lock, then it | |
| 63 // relinquishes the CMS token if it were holding that token | |
| 64 // when it acquired the low-level CMS lock. | |
| 65 // 2. any low-level notifications on the low-level lock | |
| 66 // should only be sent when a thread has relinquished the token. | |
| 67 // | |
| 68 // In the absence of either property, we'd have potential deadlock. | |
| 69 // | |
| 70 // We protect each of the CMS (concurrent and sequential) phases | |
| 71 // with the CMS _token_, not the CMS _lock_. | |
| 72 // | |
| 73 // The only code protected by CMS lock is the token acquisition code | |
| 74 // itself, see ConcurrentMarkSweepThread::[de]synchronize(), and the | |
| 75 // baton-passing code. | |
| 76 // | |
| 77 // Unfortunately, i couldn't come up with a good abstraction to factor and | |
| 78 // hide the naked CGC_lock manipulation in the baton-passing code | |
| 79 // further below. That's something we should try to do. Also, the proof | |
| 80 // of correctness of this 2-level locking scheme is far from obvious, | |
| 81 // and potentially quite slippery. We have an uneasy supsicion, for instance, | |
| 82 // that there may be a theoretical possibility of delay/starvation in the | |
| 83 // low-level lock/wait/notify scheme used for the baton-passing because of | |
| 84 // potential intereference with the priority scheme embodied in the | |
| 85 // CMS-token-passing protocol. See related comments at a CGC_lock->wait() | |
| 86 // invocation further below and marked with "XXX 20011219YSR". | |
| 87 // Indeed, as we note elsewhere, this may become yet more slippery | |
| 88 // in the presence of multiple CMS and/or multiple VM threads. XXX | |
| 89 | |
| 90 class CMSTokenSync: public StackObj { | |
| 91 private: | |
| 92 bool _is_cms_thread; | |
| 93 public: | |
| 94 CMSTokenSync(bool is_cms_thread): | |
| 95 _is_cms_thread(is_cms_thread) { | |
| 96 assert(is_cms_thread == Thread::current()->is_ConcurrentGC_thread(), | |
| 97 "Incorrect argument to constructor"); | |
| 98 ConcurrentMarkSweepThread::synchronize(_is_cms_thread); | |
| 99 } | |
| 100 | |
| 101 ~CMSTokenSync() { | |
| 102 assert(_is_cms_thread ? | |
| 103 ConcurrentMarkSweepThread::cms_thread_has_cms_token() : | |
| 104 ConcurrentMarkSweepThread::vm_thread_has_cms_token(), | |
| 105 "Incorrect state"); | |
| 106 ConcurrentMarkSweepThread::desynchronize(_is_cms_thread); | |
| 107 } | |
| 108 }; | |
| 109 | |
| 110 // Convenience class that does a CMSTokenSync, and then acquires | |
| 111 // upto three locks. | |
| 112 class CMSTokenSyncWithLocks: public CMSTokenSync { | |
| 113 private: | |
| 114 // Note: locks are acquired in textual declaration order | |
| 115 // and released in the opposite order | |
| 116 MutexLockerEx _locker1, _locker2, _locker3; | |
| 117 public: | |
| 118 CMSTokenSyncWithLocks(bool is_cms_thread, Mutex* mutex1, | |
| 119 Mutex* mutex2 = NULL, Mutex* mutex3 = NULL): | |
| 120 CMSTokenSync(is_cms_thread), | |
| 121 _locker1(mutex1, Mutex::_no_safepoint_check_flag), | |
| 122 _locker2(mutex2, Mutex::_no_safepoint_check_flag), | |
| 123 _locker3(mutex3, Mutex::_no_safepoint_check_flag) | |
| 124 { } | |
| 125 }; | |
| 126 | |
| 127 | |
| 128 // Wrapper class to temporarily disable icms during a foreground cms collection. | |
| 129 class ICMSDisabler: public StackObj { | |
| 130 public: | |
| 131 // The ctor disables icms and wakes up the thread so it notices the change; | |
| 132 // the dtor re-enables icms. Note that the CMSCollector methods will check | |
| 133 // CMSIncrementalMode. | |
| 134 ICMSDisabler() { CMSCollector::disable_icms(); CMSCollector::start_icms(); } | |
| 135 ~ICMSDisabler() { CMSCollector::enable_icms(); } | |
| 136 }; | |
| 137 | |
| 138 ////////////////////////////////////////////////////////////////// | |
| 139 // Concurrent Mark-Sweep Generation ///////////////////////////// | |
| 140 ////////////////////////////////////////////////////////////////// | |
| 141 | |
| 142 NOT_PRODUCT(CompactibleFreeListSpace* debug_cms_space;) | |
| 143 | |
| 144 // This struct contains per-thread things necessary to support parallel | |
| 145 // young-gen collection. | |
| 146 class CMSParGCThreadState: public CHeapObj { | |
| 147 public: | |
| 148 CFLS_LAB lab; | |
| 149 PromotionInfo promo; | |
| 150 | |
| 151 // Constructor. | |
| 152 CMSParGCThreadState(CompactibleFreeListSpace* cfls) : lab(cfls) { | |
| 153 promo.setSpace(cfls); | |
| 154 } | |
| 155 }; | |
| 156 | |
| 157 ConcurrentMarkSweepGeneration::ConcurrentMarkSweepGeneration( | |
| 158 ReservedSpace rs, size_t initial_byte_size, int level, | |
| 159 CardTableRS* ct, bool use_adaptive_freelists, | |
| 160 FreeBlockDictionary::DictionaryChoice dictionaryChoice) : | |
| 161 CardGeneration(rs, initial_byte_size, level, ct), | |
|
1571
2d127394260e
6916623: Align object to 16 bytes to use Compressed Oops with java heap up to 64Gb
kvn
parents:
1521
diff
changeset
|
162 _dilatation_factor(((double)MinChunkSize)/((double)(CollectedHeap::min_fill_size()))), |
| 0 | 163 _debug_collection_type(Concurrent_collection_type) |
| 164 { | |
| 165 HeapWord* bottom = (HeapWord*) _virtual_space.low(); | |
| 166 HeapWord* end = (HeapWord*) _virtual_space.high(); | |
| 167 | |
| 168 _direct_allocated_words = 0; | |
| 169 NOT_PRODUCT( | |
| 170 _numObjectsPromoted = 0; | |
| 171 _numWordsPromoted = 0; | |
| 172 _numObjectsAllocated = 0; | |
| 173 _numWordsAllocated = 0; | |
| 174 ) | |
| 175 | |
| 176 _cmsSpace = new CompactibleFreeListSpace(_bts, MemRegion(bottom, end), | |
| 177 use_adaptive_freelists, | |
| 178 dictionaryChoice); | |
| 179 NOT_PRODUCT(debug_cms_space = _cmsSpace;) | |
| 180 if (_cmsSpace == NULL) { | |
| 181 vm_exit_during_initialization( | |
| 182 "CompactibleFreeListSpace allocation failure"); | |
| 183 } | |
| 184 _cmsSpace->_gen = this; | |
| 185 | |
| 186 _gc_stats = new CMSGCStats(); | |
| 187 | |
| 188 // Verify the assumption that FreeChunk::_prev and OopDesc::_klass | |
| 189 // offsets match. The ability to tell free chunks from objects | |
| 190 // depends on this property. | |
| 191 debug_only( | |
| 192 FreeChunk* junk = NULL; | |
| 187 | 193 assert(UseCompressedOops || |
| 194 junk->prev_addr() == (void*)(oop(junk)->klass_addr()), | |
| 0 | 195 "Offset of FreeChunk::_prev within FreeChunk must match" |
| 196 " that of OopDesc::_klass within OopDesc"); | |
| 197 ) | |
| 198 if (ParallelGCThreads > 0) { | |
| 199 typedef CMSParGCThreadState* CMSParGCThreadStatePtr; | |
| 200 _par_gc_thread_states = | |
| 201 NEW_C_HEAP_ARRAY(CMSParGCThreadStatePtr, ParallelGCThreads); | |
| 202 if (_par_gc_thread_states == NULL) { | |
| 203 vm_exit_during_initialization("Could not allocate par gc structs"); | |
| 204 } | |
| 205 for (uint i = 0; i < ParallelGCThreads; i++) { | |
| 206 _par_gc_thread_states[i] = new CMSParGCThreadState(cmsSpace()); | |
| 207 if (_par_gc_thread_states[i] == NULL) { | |
| 208 vm_exit_during_initialization("Could not allocate par gc structs"); | |
| 209 } | |
| 210 } | |
| 211 } else { | |
| 212 _par_gc_thread_states = NULL; | |
| 213 } | |
| 214 _incremental_collection_failed = false; | |
| 215 // The "dilatation_factor" is the expansion that can occur on | |
| 216 // account of the fact that the minimum object size in the CMS | |
| 217 // generation may be larger than that in, say, a contiguous young | |
| 218 // generation. | |
| 219 // Ideally, in the calculation below, we'd compute the dilatation | |
| 220 // factor as: MinChunkSize/(promoting_gen's min object size) | |
| 221 // Since we do not have such a general query interface for the | |
| 222 // promoting generation, we'll instead just use the mimimum | |
| 223 // object size (which today is a header's worth of space); | |
| 224 // note that all arithmetic is in units of HeapWords. | |
|
1571
2d127394260e
6916623: Align object to 16 bytes to use Compressed Oops with java heap up to 64Gb
kvn
parents:
1521
diff
changeset
|
225 assert(MinChunkSize >= CollectedHeap::min_fill_size(), "just checking"); |
| 0 | 226 assert(_dilatation_factor >= 1.0, "from previous assert"); |
| 227 } | |
| 228 | |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
229 |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
230 // The field "_initiating_occupancy" represents the occupancy percentage |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
231 // at which we trigger a new collection cycle. Unless explicitly specified |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
232 // via CMSInitiating[Perm]OccupancyFraction (argument "io" below), it |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
233 // is calculated by: |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
234 // |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
235 // Let "f" be MinHeapFreeRatio in |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
236 // |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
237 // _intiating_occupancy = 100-f + |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
238 // f * (CMSTrigger[Perm]Ratio/100) |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
239 // where CMSTrigger[Perm]Ratio is the argument "tr" below. |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
240 // |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
241 // That is, if we assume the heap is at its desired maximum occupancy at the |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
242 // end of a collection, we let CMSTrigger[Perm]Ratio of the (purported) free |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
243 // space be allocated before initiating a new collection cycle. |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
244 // |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
245 void ConcurrentMarkSweepGeneration::init_initiating_occupancy(intx io, intx tr) { |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
246 assert(io <= 100 && tr >= 0 && tr <= 100, "Check the arguments"); |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
247 if (io >= 0) { |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
248 _initiating_occupancy = (double)io / 100.0; |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
249 } else { |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
250 _initiating_occupancy = ((100 - MinHeapFreeRatio) + |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
251 (double)(tr * MinHeapFreeRatio) / 100.0) |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
252 / 100.0; |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
253 } |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
254 } |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
255 |
| 0 | 256 void ConcurrentMarkSweepGeneration::ref_processor_init() { |
| 257 assert(collector() != NULL, "no collector"); | |
| 258 collector()->ref_processor_init(); | |
| 259 } | |
| 260 | |
| 261 void CMSCollector::ref_processor_init() { | |
| 262 if (_ref_processor == NULL) { | |
| 263 // Allocate and initialize a reference processor | |
| 264 _ref_processor = ReferenceProcessor::create_ref_processor( | |
| 265 _span, // span | |
| 266 _cmsGen->refs_discovery_is_atomic(), // atomic_discovery | |
| 267 _cmsGen->refs_discovery_is_mt(), // mt_discovery | |
| 268 &_is_alive_closure, | |
| 269 ParallelGCThreads, | |
| 270 ParallelRefProcEnabled); | |
| 271 // Initialize the _ref_processor field of CMSGen | |
| 272 _cmsGen->set_ref_processor(_ref_processor); | |
| 273 | |
| 274 // Allocate a dummy ref processor for perm gen. | |
| 275 ReferenceProcessor* rp2 = new ReferenceProcessor(); | |
| 276 if (rp2 == NULL) { | |
| 277 vm_exit_during_initialization("Could not allocate ReferenceProcessor object"); | |
| 278 } | |
| 279 _permGen->set_ref_processor(rp2); | |
| 280 } | |
| 281 } | |
| 282 | |
| 283 CMSAdaptiveSizePolicy* CMSCollector::size_policy() { | |
| 284 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 285 assert(gch->kind() == CollectedHeap::GenCollectedHeap, | |
| 286 "Wrong type of heap"); | |
| 287 CMSAdaptiveSizePolicy* sp = (CMSAdaptiveSizePolicy*) | |
| 288 gch->gen_policy()->size_policy(); | |
| 289 assert(sp->is_gc_cms_adaptive_size_policy(), | |
| 290 "Wrong type of size policy"); | |
| 291 return sp; | |
| 292 } | |
| 293 | |
| 294 CMSGCAdaptivePolicyCounters* CMSCollector::gc_adaptive_policy_counters() { | |
| 295 CMSGCAdaptivePolicyCounters* results = | |
| 296 (CMSGCAdaptivePolicyCounters*) collector_policy()->counters(); | |
| 297 assert( | |
| 298 results->kind() == GCPolicyCounters::CMSGCAdaptivePolicyCountersKind, | |
| 299 "Wrong gc policy counter kind"); | |
| 300 return results; | |
| 301 } | |
| 302 | |
| 303 | |
| 304 void ConcurrentMarkSweepGeneration::initialize_performance_counters() { | |
| 305 | |
| 306 const char* gen_name = "old"; | |
| 307 | |
| 308 // Generation Counters - generation 1, 1 subspace | |
| 309 _gen_counters = new GenerationCounters(gen_name, 1, 1, &_virtual_space); | |
| 310 | |
| 311 _space_counters = new GSpaceCounters(gen_name, 0, | |
| 312 _virtual_space.reserved_size(), | |
| 313 this, _gen_counters); | |
| 314 } | |
| 315 | |
| 316 CMSStats::CMSStats(ConcurrentMarkSweepGeneration* cms_gen, unsigned int alpha): | |
| 317 _cms_gen(cms_gen) | |
| 318 { | |
| 319 assert(alpha <= 100, "bad value"); | |
| 320 _saved_alpha = alpha; | |
| 321 | |
| 322 // Initialize the alphas to the bootstrap value of 100. | |
| 323 _gc0_alpha = _cms_alpha = 100; | |
| 324 | |
| 325 _cms_begin_time.update(); | |
| 326 _cms_end_time.update(); | |
| 327 | |
| 328 _gc0_duration = 0.0; | |
| 329 _gc0_period = 0.0; | |
| 330 _gc0_promoted = 0; | |
| 331 | |
| 332 _cms_duration = 0.0; | |
| 333 _cms_period = 0.0; | |
| 334 _cms_allocated = 0; | |
| 335 | |
| 336 _cms_used_at_gc0_begin = 0; | |
| 337 _cms_used_at_gc0_end = 0; | |
| 338 _allow_duty_cycle_reduction = false; | |
| 339 _valid_bits = 0; | |
| 340 _icms_duty_cycle = CMSIncrementalDutyCycle; | |
| 341 } | |
| 342 | |
|
1145
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
343 double CMSStats::cms_free_adjustment_factor(size_t free) const { |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
344 // TBD: CR 6909490 |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
345 return 1.0; |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
346 } |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
347 |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
348 void CMSStats::adjust_cms_free_adjustment_factor(bool fail, size_t free) { |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
349 } |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
350 |
| 0 | 351 // If promotion failure handling is on use |
| 352 // the padded average size of the promotion for each | |
| 353 // young generation collection. | |
| 354 double CMSStats::time_until_cms_gen_full() const { | |
| 355 size_t cms_free = _cms_gen->cmsSpace()->free(); | |
| 356 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 357 size_t expected_promotion = gch->get_gen(0)->capacity(); | |
| 358 if (HandlePromotionFailure) { | |
| 359 expected_promotion = MIN2( | |
| 360 (size_t) _cms_gen->gc_stats()->avg_promoted()->padded_average(), | |
| 361 expected_promotion); | |
| 362 } | |
| 363 if (cms_free > expected_promotion) { | |
| 364 // Start a cms collection if there isn't enough space to promote | |
| 365 // for the next minor collection. Use the padded average as | |
| 366 // a safety factor. | |
| 367 cms_free -= expected_promotion; | |
| 368 | |
| 369 // Adjust by the safety factor. | |
| 370 double cms_free_dbl = (double)cms_free; | |
|
1145
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
371 double cms_adjustment = (100.0 - CMSIncrementalSafetyFactor)/100.0; |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
372 // Apply a further correction factor which tries to adjust |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
373 // for recent occurance of concurrent mode failures. |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
374 cms_adjustment = cms_adjustment * cms_free_adjustment_factor(cms_free); |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
375 cms_free_dbl = cms_free_dbl * cms_adjustment; |
| 0 | 376 |
| 377 if (PrintGCDetails && Verbose) { | |
| 378 gclog_or_tty->print_cr("CMSStats::time_until_cms_gen_full: cms_free " | |
| 379 SIZE_FORMAT " expected_promotion " SIZE_FORMAT, | |
| 380 cms_free, expected_promotion); | |
| 381 gclog_or_tty->print_cr(" cms_free_dbl %f cms_consumption_rate %f", | |
| 382 cms_free_dbl, cms_consumption_rate() + 1.0); | |
| 383 } | |
| 384 // Add 1 in case the consumption rate goes to zero. | |
| 385 return cms_free_dbl / (cms_consumption_rate() + 1.0); | |
| 386 } | |
| 387 return 0.0; | |
| 388 } | |
| 389 | |
| 390 // Compare the duration of the cms collection to the | |
| 391 // time remaining before the cms generation is empty. | |
| 392 // Note that the time from the start of the cms collection | |
| 393 // to the start of the cms sweep (less than the total | |
| 394 // duration of the cms collection) can be used. This | |
| 395 // has been tried and some applications experienced | |
| 396 // promotion failures early in execution. This was | |
| 397 // possibly because the averages were not accurate | |
| 398 // enough at the beginning. | |
| 399 double CMSStats::time_until_cms_start() const { | |
| 400 // We add "gc0_period" to the "work" calculation | |
| 401 // below because this query is done (mostly) at the | |
| 402 // end of a scavenge, so we need to conservatively | |
| 403 // account for that much possible delay | |
| 404 // in the query so as to avoid concurrent mode failures | |
| 405 // due to starting the collection just a wee bit too | |
| 406 // late. | |
| 407 double work = cms_duration() + gc0_period(); | |
| 408 double deadline = time_until_cms_gen_full(); | |
|
1145
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
409 // If a concurrent mode failure occurred recently, we want to be |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
410 // more conservative and halve our expected time_until_cms_gen_full() |
| 0 | 411 if (work > deadline) { |
| 412 if (Verbose && PrintGCDetails) { | |
| 413 gclog_or_tty->print( | |
| 414 " CMSCollector: collect because of anticipated promotion " | |
| 415 "before full %3.7f + %3.7f > %3.7f ", cms_duration(), | |
| 416 gc0_period(), time_until_cms_gen_full()); | |
| 417 } | |
| 418 return 0.0; | |
| 419 } | |
| 420 return work - deadline; | |
| 421 } | |
| 422 | |
| 423 // Return a duty cycle based on old_duty_cycle and new_duty_cycle, limiting the | |
| 424 // amount of change to prevent wild oscillation. | |
| 425 unsigned int CMSStats::icms_damped_duty_cycle(unsigned int old_duty_cycle, | |
| 426 unsigned int new_duty_cycle) { | |
| 427 assert(old_duty_cycle <= 100, "bad input value"); | |
| 428 assert(new_duty_cycle <= 100, "bad input value"); | |
| 429 | |
| 430 // Note: use subtraction with caution since it may underflow (values are | |
| 431 // unsigned). Addition is safe since we're in the range 0-100. | |
| 432 unsigned int damped_duty_cycle = new_duty_cycle; | |
| 433 if (new_duty_cycle < old_duty_cycle) { | |
| 434 const unsigned int largest_delta = MAX2(old_duty_cycle / 4, 5U); | |
| 435 if (new_duty_cycle + largest_delta < old_duty_cycle) { | |
| 436 damped_duty_cycle = old_duty_cycle - largest_delta; | |
| 437 } | |
| 438 } else if (new_duty_cycle > old_duty_cycle) { | |
| 439 const unsigned int largest_delta = MAX2(old_duty_cycle / 4, 15U); | |
| 440 if (new_duty_cycle > old_duty_cycle + largest_delta) { | |
| 441 damped_duty_cycle = MIN2(old_duty_cycle + largest_delta, 100U); | |
| 442 } | |
| 443 } | |
| 444 assert(damped_duty_cycle <= 100, "invalid duty cycle computed"); | |
| 445 | |
| 446 if (CMSTraceIncrementalPacing) { | |
| 447 gclog_or_tty->print(" [icms_damped_duty_cycle(%d,%d) = %d] ", | |
| 448 old_duty_cycle, new_duty_cycle, damped_duty_cycle); | |
| 449 } | |
| 450 return damped_duty_cycle; | |
| 451 } | |
| 452 | |
| 453 unsigned int CMSStats::icms_update_duty_cycle_impl() { | |
| 454 assert(CMSIncrementalPacing && valid(), | |
| 455 "should be handled in icms_update_duty_cycle()"); | |
| 456 | |
| 457 double cms_time_so_far = cms_timer().seconds(); | |
| 458 double scaled_duration = cms_duration_per_mb() * _cms_used_at_gc0_end / M; | |
| 459 double scaled_duration_remaining = fabsd(scaled_duration - cms_time_so_far); | |
| 460 | |
| 461 // Avoid division by 0. | |
| 462 double time_until_full = MAX2(time_until_cms_gen_full(), 0.01); | |
| 463 double duty_cycle_dbl = 100.0 * scaled_duration_remaining / time_until_full; | |
| 464 | |
| 465 unsigned int new_duty_cycle = MIN2((unsigned int)duty_cycle_dbl, 100U); | |
| 466 if (new_duty_cycle > _icms_duty_cycle) { | |
| 467 // Avoid very small duty cycles (1 or 2); 0 is allowed. | |
| 468 if (new_duty_cycle > 2) { | |
| 469 _icms_duty_cycle = icms_damped_duty_cycle(_icms_duty_cycle, | |
| 470 new_duty_cycle); | |
| 471 } | |
| 472 } else if (_allow_duty_cycle_reduction) { | |
| 473 // The duty cycle is reduced only once per cms cycle (see record_cms_end()). | |
| 474 new_duty_cycle = icms_damped_duty_cycle(_icms_duty_cycle, new_duty_cycle); | |
| 475 // Respect the minimum duty cycle. | |
| 476 unsigned int min_duty_cycle = (unsigned int)CMSIncrementalDutyCycleMin; | |
| 477 _icms_duty_cycle = MAX2(new_duty_cycle, min_duty_cycle); | |
| 478 } | |
| 479 | |
| 480 if (PrintGCDetails || CMSTraceIncrementalPacing) { | |
| 481 gclog_or_tty->print(" icms_dc=%d ", _icms_duty_cycle); | |
| 482 } | |
| 483 | |
| 484 _allow_duty_cycle_reduction = false; | |
| 485 return _icms_duty_cycle; | |
| 486 } | |
| 487 | |
| 488 #ifndef PRODUCT | |
| 489 void CMSStats::print_on(outputStream *st) const { | |
| 490 st->print(" gc0_alpha=%d,cms_alpha=%d", _gc0_alpha, _cms_alpha); | |
| 491 st->print(",gc0_dur=%g,gc0_per=%g,gc0_promo=" SIZE_FORMAT, | |
| 492 gc0_duration(), gc0_period(), gc0_promoted()); | |
| 493 st->print(",cms_dur=%g,cms_dur_per_mb=%g,cms_per=%g,cms_alloc=" SIZE_FORMAT, | |
| 494 cms_duration(), cms_duration_per_mb(), | |
| 495 cms_period(), cms_allocated()); | |
| 496 st->print(",cms_since_beg=%g,cms_since_end=%g", | |
| 497 cms_time_since_begin(), cms_time_since_end()); | |
| 498 st->print(",cms_used_beg=" SIZE_FORMAT ",cms_used_end=" SIZE_FORMAT, | |
| 499 _cms_used_at_gc0_begin, _cms_used_at_gc0_end); | |
| 500 if (CMSIncrementalMode) { | |
| 501 st->print(",dc=%d", icms_duty_cycle()); | |
| 502 } | |
| 503 | |
| 504 if (valid()) { | |
| 505 st->print(",promo_rate=%g,cms_alloc_rate=%g", | |
| 506 promotion_rate(), cms_allocation_rate()); | |
| 507 st->print(",cms_consumption_rate=%g,time_until_full=%g", | |
| 508 cms_consumption_rate(), time_until_cms_gen_full()); | |
| 509 } | |
| 510 st->print(" "); | |
| 511 } | |
| 512 #endif // #ifndef PRODUCT | |
| 513 | |
| 514 CMSCollector::CollectorState CMSCollector::_collectorState = | |
| 515 CMSCollector::Idling; | |
| 516 bool CMSCollector::_foregroundGCIsActive = false; | |
| 517 bool CMSCollector::_foregroundGCShouldWait = false; | |
| 518 | |
| 519 CMSCollector::CMSCollector(ConcurrentMarkSweepGeneration* cmsGen, | |
| 520 ConcurrentMarkSweepGeneration* permGen, | |
| 521 CardTableRS* ct, | |
| 522 ConcurrentMarkSweepPolicy* cp): | |
| 523 _cmsGen(cmsGen), | |
| 524 _permGen(permGen), | |
| 525 _ct(ct), | |
| 526 _ref_processor(NULL), // will be set later | |
| 527 _conc_workers(NULL), // may be set later | |
| 528 _abort_preclean(false), | |
| 529 _start_sampling(false), | |
| 530 _between_prologue_and_epilogue(false), | |
| 531 _markBitMap(0, Mutex::leaf + 1, "CMS_markBitMap_lock"), | |
| 532 _perm_gen_verify_bit_map(0, -1 /* no mutex */, "No_lock"), | |
| 533 _modUnionTable((CardTableModRefBS::card_shift - LogHeapWordSize), | |
| 534 -1 /* lock-free */, "No_lock" /* dummy */), | |
| 535 _modUnionClosure(&_modUnionTable), | |
| 536 _modUnionClosurePar(&_modUnionTable), | |
|
143
b5489bb705c9
6662086: 6u4+, 7b11+: CMS never clears referents when -XX:+ParallelRefProcEnabled
ysr
parents:
113
diff
changeset
|
537 // Adjust my span to cover old (cms) gen and perm gen |
|
b5489bb705c9
6662086: 6u4+, 7b11+: CMS never clears referents when -XX:+ParallelRefProcEnabled
ysr
parents:
113
diff
changeset
|
538 _span(cmsGen->reserved()._union(permGen->reserved())), |
|
b5489bb705c9
6662086: 6u4+, 7b11+: CMS never clears referents when -XX:+ParallelRefProcEnabled
ysr
parents:
113
diff
changeset
|
539 // Construct the is_alive_closure with _span & markBitMap |
|
b5489bb705c9
6662086: 6u4+, 7b11+: CMS never clears referents when -XX:+ParallelRefProcEnabled
ysr
parents:
113
diff
changeset
|
540 _is_alive_closure(_span, &_markBitMap), |
| 0 | 541 _restart_addr(NULL), |
| 542 _overflow_list(NULL), | |
| 543 _preserved_oop_stack(NULL), | |
| 544 _preserved_mark_stack(NULL), | |
| 545 _stats(cmsGen), | |
| 546 _eden_chunk_array(NULL), // may be set in ctor body | |
| 547 _eden_chunk_capacity(0), // -- ditto -- | |
| 548 _eden_chunk_index(0), // -- ditto -- | |
| 549 _survivor_plab_array(NULL), // -- ditto -- | |
| 550 _survivor_chunk_array(NULL), // -- ditto -- | |
| 551 _survivor_chunk_capacity(0), // -- ditto -- | |
| 552 _survivor_chunk_index(0), // -- ditto -- | |
| 553 _ser_pmc_preclean_ovflw(0), | |
|
452
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
554 _ser_kac_preclean_ovflw(0), |
| 0 | 555 _ser_pmc_remark_ovflw(0), |
| 556 _par_pmc_remark_ovflw(0), | |
| 557 _ser_kac_ovflw(0), | |
| 558 _par_kac_ovflw(0), | |
| 559 #ifndef PRODUCT | |
| 560 _num_par_pushes(0), | |
| 561 #endif | |
| 562 _collection_count_start(0), | |
| 563 _verifying(false), | |
| 564 _icms_start_limit(NULL), | |
| 565 _icms_stop_limit(NULL), | |
| 566 _verification_mark_bm(0, Mutex::leaf + 1, "CMS_verification_mark_bm_lock"), | |
| 567 _completed_initialization(false), | |
| 568 _collector_policy(cp), | |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
569 _should_unload_classes(false), |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
570 _concurrent_cycles_since_last_unload(0), |
|
798
fe1574da39fc
6848641: CMSCollector::_roots_scanning_options should be initialized
ysr
parents:
679
diff
changeset
|
571 _roots_scanning_options(0), |
|
1145
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
572 _inter_sweep_estimate(CMS_SweepWeight, CMS_SweepPadding), |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
573 _intra_sweep_estimate(CMS_SweepWeight, CMS_SweepPadding) |
| 0 | 574 { |
| 575 if (ExplicitGCInvokesConcurrentAndUnloadsClasses) { | |
| 576 ExplicitGCInvokesConcurrent = true; | |
| 577 } | |
| 578 // Now expand the span and allocate the collection support structures | |
| 579 // (MUT, marking bit map etc.) to cover both generations subject to | |
| 580 // collection. | |
| 581 | |
| 582 // First check that _permGen is adjacent to _cmsGen and above it. | |
| 583 assert( _cmsGen->reserved().word_size() > 0 | |
| 584 && _permGen->reserved().word_size() > 0, | |
| 585 "generations should not be of zero size"); | |
| 586 assert(_cmsGen->reserved().intersection(_permGen->reserved()).is_empty(), | |
| 587 "_cmsGen and _permGen should not overlap"); | |
| 588 assert(_cmsGen->reserved().end() == _permGen->reserved().start(), | |
| 589 "_cmsGen->end() different from _permGen->start()"); | |
| 590 | |
| 591 // For use by dirty card to oop closures. | |
| 592 _cmsGen->cmsSpace()->set_collector(this); | |
| 593 _permGen->cmsSpace()->set_collector(this); | |
| 594 | |
| 595 // Allocate MUT and marking bit map | |
| 596 { | |
| 597 MutexLockerEx x(_markBitMap.lock(), Mutex::_no_safepoint_check_flag); | |
| 598 if (!_markBitMap.allocate(_span)) { | |
| 599 warning("Failed to allocate CMS Bit Map"); | |
| 600 return; | |
| 601 } | |
| 602 assert(_markBitMap.covers(_span), "_markBitMap inconsistency?"); | |
| 603 } | |
| 604 { | |
| 605 _modUnionTable.allocate(_span); | |
| 606 assert(_modUnionTable.covers(_span), "_modUnionTable inconsistency?"); | |
| 607 } | |
| 608 | |
| 1284 | 609 if (!_markStack.allocate(MarkStackSize)) { |
| 0 | 610 warning("Failed to allocate CMS Marking Stack"); |
| 611 return; | |
| 612 } | |
| 613 if (!_revisitStack.allocate(CMSRevisitStackSize)) { | |
| 614 warning("Failed to allocate CMS Revisit Stack"); | |
| 615 return; | |
| 616 } | |
| 617 | |
| 618 // Support for multi-threaded concurrent phases | |
| 619 if (ParallelGCThreads > 0 && CMSConcurrentMTEnabled) { | |
| 1284 | 620 if (FLAG_IS_DEFAULT(ConcGCThreads)) { |
| 0 | 621 // just for now |
| 1284 | 622 FLAG_SET_DEFAULT(ConcGCThreads, (ParallelGCThreads + 3)/4); |
| 623 } | |
| 624 if (ConcGCThreads > 1) { | |
| 0 | 625 _conc_workers = new YieldingFlexibleWorkGang("Parallel CMS Threads", |
| 1284 | 626 ConcGCThreads, true); |
| 0 | 627 if (_conc_workers == NULL) { |
| 628 warning("GC/CMS: _conc_workers allocation failure: " | |
| 629 "forcing -CMSConcurrentMTEnabled"); | |
| 630 CMSConcurrentMTEnabled = false; | |
| 631 } | |
| 632 } else { | |
| 633 CMSConcurrentMTEnabled = false; | |
| 634 } | |
| 635 } | |
| 636 if (!CMSConcurrentMTEnabled) { | |
| 1284 | 637 ConcGCThreads = 0; |
| 0 | 638 } else { |
| 639 // Turn off CMSCleanOnEnter optimization temporarily for | |
| 640 // the MT case where it's not fixed yet; see 6178663. | |
| 641 CMSCleanOnEnter = false; | |
| 642 } | |
| 1284 | 643 assert((_conc_workers != NULL) == (ConcGCThreads > 1), |
| 0 | 644 "Inconsistency"); |
| 645 | |
| 646 // Parallel task queues; these are shared for the | |
| 647 // concurrent and stop-world phases of CMS, but | |
| 648 // are not shared with parallel scavenge (ParNew). | |
| 649 { | |
| 650 uint i; | |
| 1284 | 651 uint num_queues = (uint) MAX2(ParallelGCThreads, ConcGCThreads); |
| 0 | 652 |
| 653 if ((CMSParallelRemarkEnabled || CMSConcurrentMTEnabled | |
| 654 || ParallelRefProcEnabled) | |
| 655 && num_queues > 0) { | |
| 656 _task_queues = new OopTaskQueueSet(num_queues); | |
| 657 if (_task_queues == NULL) { | |
| 658 warning("task_queues allocation failure."); | |
| 659 return; | |
| 660 } | |
| 661 _hash_seed = NEW_C_HEAP_ARRAY(int, num_queues); | |
| 662 if (_hash_seed == NULL) { | |
| 663 warning("_hash_seed array allocation failure"); | |
| 664 return; | |
| 665 } | |
| 666 | |
| 1665 | 667 typedef Padded<OopTaskQueue> PaddedOopTaskQueue; |
| 0 | 668 for (i = 0; i < num_queues; i++) { |
| 1665 | 669 PaddedOopTaskQueue *q = new PaddedOopTaskQueue(); |
| 670 if (q == NULL) { | |
| 0 | 671 warning("work_queue allocation failure."); |
| 672 return; | |
| 673 } | |
| 1665 | 674 _task_queues->register_queue(i, q); |
| 0 | 675 } |
| 676 for (i = 0; i < num_queues; i++) { | |
| 677 _task_queues->queue(i)->initialize(); | |
| 678 _hash_seed[i] = 17; // copied from ParNew | |
| 679 } | |
| 680 } | |
| 681 } | |
| 682 | |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
683 _cmsGen ->init_initiating_occupancy(CMSInitiatingOccupancyFraction, CMSTriggerRatio); |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
684 _permGen->init_initiating_occupancy(CMSInitiatingPermOccupancyFraction, CMSTriggerPermRatio); |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
685 |
| 0 | 686 // Clip CMSBootstrapOccupancy between 0 and 100. |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
687 _bootstrap_occupancy = ((double)MIN2((uintx)100, MAX2((uintx)0, CMSBootstrapOccupancy))) |
| 0 | 688 /(double)100; |
| 689 | |
| 690 _full_gcs_since_conc_gc = 0; | |
| 691 | |
| 692 // Now tell CMS generations the identity of their collector | |
| 693 ConcurrentMarkSweepGeneration::set_collector(this); | |
| 694 | |
| 695 // Create & start a CMS thread for this CMS collector | |
| 696 _cmsThread = ConcurrentMarkSweepThread::start(this); | |
| 697 assert(cmsThread() != NULL, "CMS Thread should have been created"); | |
| 698 assert(cmsThread()->collector() == this, | |
| 699 "CMS Thread should refer to this gen"); | |
| 700 assert(CGC_lock != NULL, "Where's the CGC_lock?"); | |
| 701 | |
| 702 // Support for parallelizing young gen rescan | |
| 703 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 704 _young_gen = gch->prev_gen(_cmsGen); | |
| 705 if (gch->supports_inline_contig_alloc()) { | |
| 706 _top_addr = gch->top_addr(); | |
| 707 _end_addr = gch->end_addr(); | |
| 708 assert(_young_gen != NULL, "no _young_gen"); | |
| 709 _eden_chunk_index = 0; | |
| 710 _eden_chunk_capacity = (_young_gen->max_capacity()+CMSSamplingGrain)/CMSSamplingGrain; | |
| 711 _eden_chunk_array = NEW_C_HEAP_ARRAY(HeapWord*, _eden_chunk_capacity); | |
| 712 if (_eden_chunk_array == NULL) { | |
| 713 _eden_chunk_capacity = 0; | |
| 714 warning("GC/CMS: _eden_chunk_array allocation failure"); | |
| 715 } | |
| 716 } | |
| 717 assert(_eden_chunk_array != NULL || _eden_chunk_capacity == 0, "Error"); | |
| 718 | |
| 719 // Support for parallelizing survivor space rescan | |
| 720 if (CMSParallelRemarkEnabled && CMSParallelSurvivorRemarkEnabled) { | |
|
1289
d47555d7aca8
6910182: CMS: assert(_cursor[j] == _survivor_plab_array[j].end(),"Ctl pt invariant")
jmasa
parents:
1284
diff
changeset
|
721 const size_t max_plab_samples = |
|
d47555d7aca8
6910182: CMS: assert(_cursor[j] == _survivor_plab_array[j].end(),"Ctl pt invariant")
jmasa
parents:
1284
diff
changeset
|
722 ((DefNewGeneration*)_young_gen)->max_survivor_size()/MinTLABSize; |
|
d47555d7aca8
6910182: CMS: assert(_cursor[j] == _survivor_plab_array[j].end(),"Ctl pt invariant")
jmasa
parents:
1284
diff
changeset
|
723 |
| 0 | 724 _survivor_plab_array = NEW_C_HEAP_ARRAY(ChunkArray, ParallelGCThreads); |
| 725 _survivor_chunk_array = NEW_C_HEAP_ARRAY(HeapWord*, 2*max_plab_samples); | |
| 726 _cursor = NEW_C_HEAP_ARRAY(size_t, ParallelGCThreads); | |
| 727 if (_survivor_plab_array == NULL || _survivor_chunk_array == NULL | |
| 728 || _cursor == NULL) { | |
| 729 warning("Failed to allocate survivor plab/chunk array"); | |
| 730 if (_survivor_plab_array != NULL) { | |
| 731 FREE_C_HEAP_ARRAY(ChunkArray, _survivor_plab_array); | |
| 732 _survivor_plab_array = NULL; | |
| 733 } | |
| 734 if (_survivor_chunk_array != NULL) { | |
| 735 FREE_C_HEAP_ARRAY(HeapWord*, _survivor_chunk_array); | |
| 736 _survivor_chunk_array = NULL; | |
| 737 } | |
| 738 if (_cursor != NULL) { | |
| 739 FREE_C_HEAP_ARRAY(size_t, _cursor); | |
| 740 _cursor = NULL; | |
| 741 } | |
| 742 } else { | |
| 743 _survivor_chunk_capacity = 2*max_plab_samples; | |
| 744 for (uint i = 0; i < ParallelGCThreads; i++) { | |
| 745 HeapWord** vec = NEW_C_HEAP_ARRAY(HeapWord*, max_plab_samples); | |
| 746 if (vec == NULL) { | |
| 747 warning("Failed to allocate survivor plab array"); | |
| 748 for (int j = i; j > 0; j--) { | |
| 749 FREE_C_HEAP_ARRAY(HeapWord*, _survivor_plab_array[j-1].array()); | |
| 750 } | |
| 751 FREE_C_HEAP_ARRAY(ChunkArray, _survivor_plab_array); | |
| 752 FREE_C_HEAP_ARRAY(HeapWord*, _survivor_chunk_array); | |
| 753 _survivor_plab_array = NULL; | |
| 754 _survivor_chunk_array = NULL; | |
| 755 _survivor_chunk_capacity = 0; | |
| 756 break; | |
| 757 } else { | |
| 758 ChunkArray* cur = | |
| 759 ::new (&_survivor_plab_array[i]) ChunkArray(vec, | |
| 760 max_plab_samples); | |
| 761 assert(cur->end() == 0, "Should be 0"); | |
| 762 assert(cur->array() == vec, "Should be vec"); | |
| 763 assert(cur->capacity() == max_plab_samples, "Error"); | |
| 764 } | |
| 765 } | |
| 766 } | |
| 767 } | |
| 768 assert( ( _survivor_plab_array != NULL | |
| 769 && _survivor_chunk_array != NULL) | |
| 770 || ( _survivor_chunk_capacity == 0 | |
| 771 && _survivor_chunk_index == 0), | |
| 772 "Error"); | |
| 773 | |
| 774 // Choose what strong roots should be scanned depending on verification options | |
| 775 // and perm gen collection mode. | |
| 776 if (!CMSClassUnloadingEnabled) { | |
| 777 // If class unloading is disabled we want to include all classes into the root set. | |
| 778 add_root_scanning_option(SharedHeap::SO_AllClasses); | |
| 779 } else { | |
| 780 add_root_scanning_option(SharedHeap::SO_SystemClasses); | |
| 781 } | |
| 782 | |
| 783 NOT_PRODUCT(_overflow_counter = CMSMarkStackOverflowInterval;) | |
| 784 _gc_counters = new CollectorCounters("CMS", 1); | |
| 785 _completed_initialization = true; | |
|
1145
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
786 _inter_sweep_timer.start(); // start of time |
|
1518
3bfae429e2cf
6948537: CMS: BOT walkers observe out-of-thin-air zeros on sun4v sparc/CMT
ysr
parents:
1387
diff
changeset
|
787 #ifdef SPARC |
|
3bfae429e2cf
6948537: CMS: BOT walkers observe out-of-thin-air zeros on sun4v sparc/CMT
ysr
parents:
1387
diff
changeset
|
788 // Issue a stern warning, but allow use for experimentation and debugging. |
|
3bfae429e2cf
6948537: CMS: BOT walkers observe out-of-thin-air zeros on sun4v sparc/CMT
ysr
parents:
1387
diff
changeset
|
789 if (VM_Version::is_sun4v() && UseMemSetInBOT) { |
|
3bfae429e2cf
6948537: CMS: BOT walkers observe out-of-thin-air zeros on sun4v sparc/CMT
ysr
parents:
1387
diff
changeset
|
790 assert(!FLAG_IS_DEFAULT(UseMemSetInBOT), "Error"); |
|
3bfae429e2cf
6948537: CMS: BOT walkers observe out-of-thin-air zeros on sun4v sparc/CMT
ysr
parents:
1387
diff
changeset
|
791 warning("Experimental flag -XX:+UseMemSetInBOT is known to cause instability" |
|
3bfae429e2cf
6948537: CMS: BOT walkers observe out-of-thin-air zeros on sun4v sparc/CMT
ysr
parents:
1387
diff
changeset
|
792 " on sun4v; please understand that you are using at your own risk!"); |
|
3bfae429e2cf
6948537: CMS: BOT walkers observe out-of-thin-air zeros on sun4v sparc/CMT
ysr
parents:
1387
diff
changeset
|
793 } |
|
3bfae429e2cf
6948537: CMS: BOT walkers observe out-of-thin-air zeros on sun4v sparc/CMT
ysr
parents:
1387
diff
changeset
|
794 #endif |
| 0 | 795 } |
| 796 | |
| 797 const char* ConcurrentMarkSweepGeneration::name() const { | |
| 798 return "concurrent mark-sweep generation"; | |
| 799 } | |
| 800 void ConcurrentMarkSweepGeneration::update_counters() { | |
| 801 if (UsePerfData) { | |
| 802 _space_counters->update_all(); | |
| 803 _gen_counters->update_all(); | |
| 804 } | |
| 805 } | |
| 806 | |
| 807 // this is an optimized version of update_counters(). it takes the | |
| 808 // used value as a parameter rather than computing it. | |
| 809 // | |
| 810 void ConcurrentMarkSweepGeneration::update_counters(size_t used) { | |
| 811 if (UsePerfData) { | |
| 812 _space_counters->update_used(used); | |
| 813 _space_counters->update_capacity(); | |
| 814 _gen_counters->update_all(); | |
| 815 } | |
| 816 } | |
| 817 | |
| 818 void ConcurrentMarkSweepGeneration::print() const { | |
| 819 Generation::print(); | |
| 820 cmsSpace()->print(); | |
| 821 } | |
| 822 | |
| 823 #ifndef PRODUCT | |
| 824 void ConcurrentMarkSweepGeneration::print_statistics() { | |
| 825 cmsSpace()->printFLCensus(0); | |
| 826 } | |
| 827 #endif | |
| 828 | |
| 829 void ConcurrentMarkSweepGeneration::printOccupancy(const char *s) { | |
| 830 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 831 if (PrintGCDetails) { | |
| 832 if (Verbose) { | |
| 833 gclog_or_tty->print(" [%d %s-%s: "SIZE_FORMAT"("SIZE_FORMAT")]", | |
| 834 level(), short_name(), s, used(), capacity()); | |
| 835 } else { | |
| 836 gclog_or_tty->print(" [%d %s-%s: "SIZE_FORMAT"K("SIZE_FORMAT"K)]", | |
| 837 level(), short_name(), s, used() / K, capacity() / K); | |
| 838 } | |
| 839 } | |
| 840 if (Verbose) { | |
| 841 gclog_or_tty->print(" "SIZE_FORMAT"("SIZE_FORMAT")", | |
| 842 gch->used(), gch->capacity()); | |
| 843 } else { | |
| 844 gclog_or_tty->print(" "SIZE_FORMAT"K("SIZE_FORMAT"K)", | |
| 845 gch->used() / K, gch->capacity() / K); | |
| 846 } | |
| 847 } | |
| 848 | |
| 849 size_t | |
| 850 ConcurrentMarkSweepGeneration::contiguous_available() const { | |
| 851 // dld proposes an improvement in precision here. If the committed | |
| 852 // part of the space ends in a free block we should add that to | |
| 853 // uncommitted size in the calculation below. Will make this | |
| 854 // change later, staying with the approximation below for the | |
| 855 // time being. -- ysr. | |
| 856 return MAX2(_virtual_space.uncommitted_size(), unsafe_max_alloc_nogc()); | |
| 857 } | |
| 858 | |
| 859 size_t | |
| 860 ConcurrentMarkSweepGeneration::unsafe_max_alloc_nogc() const { | |
| 861 return _cmsSpace->max_alloc_in_words() * HeapWordSize; | |
| 862 } | |
| 863 | |
| 864 size_t ConcurrentMarkSweepGeneration::max_available() const { | |
| 865 return free() + _virtual_space.uncommitted_size(); | |
| 866 } | |
| 867 | |
| 868 bool ConcurrentMarkSweepGeneration::promotion_attempt_is_safe( | |
| 869 size_t max_promotion_in_bytes, | |
| 870 bool younger_handles_promotion_failure) const { | |
| 871 | |
| 872 // This is the most conservative test. Full promotion is | |
| 873 // guaranteed if this is used. The multiplicative factor is to | |
| 874 // account for the worst case "dilatation". | |
| 875 double adjusted_max_promo_bytes = _dilatation_factor * max_promotion_in_bytes; | |
| 876 if (adjusted_max_promo_bytes > (double)max_uintx) { // larger than size_t | |
| 877 adjusted_max_promo_bytes = (double)max_uintx; | |
| 878 } | |
| 879 bool result = (max_contiguous_available() >= (size_t)adjusted_max_promo_bytes); | |
| 880 | |
| 881 if (younger_handles_promotion_failure && !result) { | |
| 882 // Full promotion is not guaranteed because fragmentation | |
| 883 // of the cms generation can prevent the full promotion. | |
| 884 result = (max_available() >= (size_t)adjusted_max_promo_bytes); | |
| 885 | |
| 886 if (!result) { | |
| 887 // With promotion failure handling the test for the ability | |
| 888 // to support the promotion does not have to be guaranteed. | |
| 889 // Use an average of the amount promoted. | |
| 890 result = max_available() >= (size_t) | |
| 891 gc_stats()->avg_promoted()->padded_average(); | |
| 892 if (PrintGC && Verbose && result) { | |
| 893 gclog_or_tty->print_cr( | |
| 894 "\nConcurrentMarkSweepGeneration::promotion_attempt_is_safe" | |
| 895 " max_available: " SIZE_FORMAT | |
| 896 " avg_promoted: " SIZE_FORMAT, | |
| 897 max_available(), (size_t) | |
| 898 gc_stats()->avg_promoted()->padded_average()); | |
| 899 } | |
| 900 } else { | |
| 901 if (PrintGC && Verbose) { | |
| 902 gclog_or_tty->print_cr( | |
| 903 "\nConcurrentMarkSweepGeneration::promotion_attempt_is_safe" | |
| 904 " max_available: " SIZE_FORMAT | |
| 905 " adj_max_promo_bytes: " SIZE_FORMAT, | |
| 906 max_available(), (size_t)adjusted_max_promo_bytes); | |
| 907 } | |
| 908 } | |
| 909 } else { | |
| 910 if (PrintGC && Verbose) { | |
| 911 gclog_or_tty->print_cr( | |
| 912 "\nConcurrentMarkSweepGeneration::promotion_attempt_is_safe" | |
| 913 " contiguous_available: " SIZE_FORMAT | |
| 914 " adj_max_promo_bytes: " SIZE_FORMAT, | |
| 915 max_contiguous_available(), (size_t)adjusted_max_promo_bytes); | |
| 916 } | |
| 917 } | |
| 918 return result; | |
| 919 } | |
| 920 | |
|
1145
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
921 // At a promotion failure dump information on block layout in heap |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
922 // (cms old generation). |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
923 void ConcurrentMarkSweepGeneration::promotion_failure_occurred() { |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
924 if (CMSDumpAtPromotionFailure) { |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
925 cmsSpace()->dump_at_safepoint_with_locks(collector(), gclog_or_tty); |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
926 } |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
927 } |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
928 |
| 0 | 929 CompactibleSpace* |
| 930 ConcurrentMarkSweepGeneration::first_compaction_space() const { | |
| 931 return _cmsSpace; | |
| 932 } | |
| 933 | |
| 934 void ConcurrentMarkSweepGeneration::reset_after_compaction() { | |
| 935 // Clear the promotion information. These pointers can be adjusted | |
| 936 // along with all the other pointers into the heap but | |
| 937 // compaction is expected to be a rare event with | |
| 938 // a heap using cms so don't do it without seeing the need. | |
| 939 if (ParallelGCThreads > 0) { | |
| 940 for (uint i = 0; i < ParallelGCThreads; i++) { | |
| 941 _par_gc_thread_states[i]->promo.reset(); | |
| 942 } | |
| 943 } | |
| 944 } | |
| 945 | |
| 946 void ConcurrentMarkSweepGeneration::space_iterate(SpaceClosure* blk, bool usedOnly) { | |
| 947 blk->do_space(_cmsSpace); | |
| 948 } | |
| 949 | |
| 950 void ConcurrentMarkSweepGeneration::compute_new_size() { | |
| 951 assert_locked_or_safepoint(Heap_lock); | |
| 952 | |
| 953 // If incremental collection failed, we just want to expand | |
| 954 // to the limit. | |
| 955 if (incremental_collection_failed()) { | |
| 956 clear_incremental_collection_failed(); | |
| 957 grow_to_reserved(); | |
| 958 return; | |
| 959 } | |
| 960 | |
| 961 size_t expand_bytes = 0; | |
| 962 double free_percentage = ((double) free()) / capacity(); | |
| 963 double desired_free_percentage = (double) MinHeapFreeRatio / 100; | |
| 964 double maximum_free_percentage = (double) MaxHeapFreeRatio / 100; | |
| 965 | |
| 966 // compute expansion delta needed for reaching desired free percentage | |
| 967 if (free_percentage < desired_free_percentage) { | |
| 968 size_t desired_capacity = (size_t)(used() / ((double) 1 - desired_free_percentage)); | |
| 969 assert(desired_capacity >= capacity(), "invalid expansion size"); | |
| 970 expand_bytes = MAX2(desired_capacity - capacity(), MinHeapDeltaBytes); | |
| 971 } | |
| 972 if (expand_bytes > 0) { | |
| 973 if (PrintGCDetails && Verbose) { | |
| 974 size_t desired_capacity = (size_t)(used() / ((double) 1 - desired_free_percentage)); | |
| 975 gclog_or_tty->print_cr("\nFrom compute_new_size: "); | |
| 976 gclog_or_tty->print_cr(" Free fraction %f", free_percentage); | |
| 977 gclog_or_tty->print_cr(" Desired free fraction %f", | |
| 978 desired_free_percentage); | |
| 979 gclog_or_tty->print_cr(" Maximum free fraction %f", | |
| 980 maximum_free_percentage); | |
| 981 gclog_or_tty->print_cr(" Capactiy "SIZE_FORMAT, capacity()/1000); | |
| 982 gclog_or_tty->print_cr(" Desired capacity "SIZE_FORMAT, | |
| 983 desired_capacity/1000); | |
| 984 int prev_level = level() - 1; | |
| 985 if (prev_level >= 0) { | |
| 986 size_t prev_size = 0; | |
| 987 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 988 Generation* prev_gen = gch->_gens[prev_level]; | |
| 989 prev_size = prev_gen->capacity(); | |
| 990 gclog_or_tty->print_cr(" Younger gen size "SIZE_FORMAT, | |
| 991 prev_size/1000); | |
| 992 } | |
| 993 gclog_or_tty->print_cr(" unsafe_max_alloc_nogc "SIZE_FORMAT, | |
| 994 unsafe_max_alloc_nogc()/1000); | |
| 995 gclog_or_tty->print_cr(" contiguous available "SIZE_FORMAT, | |
| 996 contiguous_available()/1000); | |
| 997 gclog_or_tty->print_cr(" Expand by "SIZE_FORMAT" (bytes)", | |
| 998 expand_bytes); | |
| 999 } | |
| 1000 // safe if expansion fails | |
| 1001 expand(expand_bytes, 0, CMSExpansionCause::_satisfy_free_ratio); | |
| 1002 if (PrintGCDetails && Verbose) { | |
| 1003 gclog_or_tty->print_cr(" Expanded free fraction %f", | |
| 1004 ((double) free()) / capacity()); | |
| 1005 } | |
| 1006 } | |
| 1007 } | |
| 1008 | |
| 1009 Mutex* ConcurrentMarkSweepGeneration::freelistLock() const { | |
| 1010 return cmsSpace()->freelistLock(); | |
| 1011 } | |
| 1012 | |
| 1013 HeapWord* ConcurrentMarkSweepGeneration::allocate(size_t size, | |
| 1014 bool tlab) { | |
| 1015 CMSSynchronousYieldRequest yr; | |
| 1016 MutexLockerEx x(freelistLock(), | |
| 1017 Mutex::_no_safepoint_check_flag); | |
| 1018 return have_lock_and_allocate(size, tlab); | |
| 1019 } | |
| 1020 | |
| 1021 HeapWord* ConcurrentMarkSweepGeneration::have_lock_and_allocate(size_t size, | |
|
1716
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1022 bool tlab /* ignored */) { |
| 0 | 1023 assert_lock_strong(freelistLock()); |
| 1024 size_t adjustedSize = CompactibleFreeListSpace::adjustObjectSize(size); | |
| 1025 HeapWord* res = cmsSpace()->allocate(adjustedSize); | |
| 1026 // Allocate the object live (grey) if the background collector has | |
| 1027 // started marking. This is necessary because the marker may | |
| 1028 // have passed this address and consequently this object will | |
| 1029 // not otherwise be greyed and would be incorrectly swept up. | |
| 1030 // Note that if this object contains references, the writing | |
| 1031 // of those references will dirty the card containing this object | |
| 1032 // allowing the object to be blackened (and its references scanned) | |
| 1033 // either during a preclean phase or at the final checkpoint. | |
| 1034 if (res != NULL) { | |
|
1716
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1035 // We may block here with an uninitialized object with |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1036 // its mark-bit or P-bits not yet set. Such objects need |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1037 // to be safely navigable by block_start(). |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1038 assert(oop(res)->klass_or_null() == NULL, "Object should be uninitialized here."); |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1039 assert(!((FreeChunk*)res)->isFree(), "Error, block will look free but show wrong size"); |
| 0 | 1040 collector()->direct_allocated(res, adjustedSize); |
| 1041 _direct_allocated_words += adjustedSize; | |
| 1042 // allocation counters | |
| 1043 NOT_PRODUCT( | |
| 1044 _numObjectsAllocated++; | |
| 1045 _numWordsAllocated += (int)adjustedSize; | |
| 1046 ) | |
| 1047 } | |
| 1048 return res; | |
| 1049 } | |
| 1050 | |
| 1051 // In the case of direct allocation by mutators in a generation that | |
| 1052 // is being concurrently collected, the object must be allocated | |
| 1053 // live (grey) if the background collector has started marking. | |
| 1054 // This is necessary because the marker may | |
| 1055 // have passed this address and consequently this object will | |
| 1056 // not otherwise be greyed and would be incorrectly swept up. | |
| 1057 // Note that if this object contains references, the writing | |
| 1058 // of those references will dirty the card containing this object | |
| 1059 // allowing the object to be blackened (and its references scanned) | |
| 1060 // either during a preclean phase or at the final checkpoint. | |
| 1061 void CMSCollector::direct_allocated(HeapWord* start, size_t size) { | |
| 1062 assert(_markBitMap.covers(start, size), "Out of bounds"); | |
| 1063 if (_collectorState >= Marking) { | |
| 1064 MutexLockerEx y(_markBitMap.lock(), | |
| 1065 Mutex::_no_safepoint_check_flag); | |
| 1066 // [see comments preceding SweepClosure::do_blk() below for details] | |
| 1067 // 1. need to mark the object as live so it isn't collected | |
| 1068 // 2. need to mark the 2nd bit to indicate the object may be uninitialized | |
|
1716
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1069 // 3. need to mark the end of the object so marking, precleaning or sweeping |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1070 // can skip over uninitialized or unparsable objects. An allocated |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1071 // object is considered uninitialized for our purposes as long as |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1072 // its klass word is NULL. (Unparsable objects are those which are |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1073 // initialized in the sense just described, but whose sizes can still |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1074 // not be correctly determined. Note that the class of unparsable objects |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1075 // can only occur in the perm gen. All old gen objects are parsable |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1076 // as soon as they are initialized.) |
| 0 | 1077 _markBitMap.mark(start); // object is live |
| 1078 _markBitMap.mark(start + 1); // object is potentially uninitialized? | |
| 1079 _markBitMap.mark(start + size - 1); | |
| 1080 // mark end of object | |
| 1081 } | |
| 1082 // check that oop looks uninitialized | |
| 187 | 1083 assert(oop(start)->klass_or_null() == NULL, "_klass should be NULL"); |
| 0 | 1084 } |
| 1085 | |
| 1086 void CMSCollector::promoted(bool par, HeapWord* start, | |
| 1087 bool is_obj_array, size_t obj_size) { | |
| 1088 assert(_markBitMap.covers(start), "Out of bounds"); | |
| 1089 // See comment in direct_allocated() about when objects should | |
| 1090 // be allocated live. | |
| 1091 if (_collectorState >= Marking) { | |
| 1092 // we already hold the marking bit map lock, taken in | |
| 1093 // the prologue | |
| 1094 if (par) { | |
| 1095 _markBitMap.par_mark(start); | |
| 1096 } else { | |
| 1097 _markBitMap.mark(start); | |
| 1098 } | |
| 1099 // We don't need to mark the object as uninitialized (as | |
| 1100 // in direct_allocated above) because this is being done with the | |
| 1101 // world stopped and the object will be initialized by the | |
|
1716
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1102 // time the marking, precleaning or sweeping get to look at it. |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1103 // But see the code for copying objects into the CMS generation, |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1104 // where we need to ensure that concurrent readers of the |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1105 // block offset table are able to safely navigate a block that |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1106 // is in flux from being free to being allocated (and in |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1107 // transition while being copied into) and subsequently |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1108 // becoming a bona-fide object when the copy/promotion is complete. |
| 0 | 1109 assert(SafepointSynchronize::is_at_safepoint(), |
| 1110 "expect promotion only at safepoints"); | |
| 1111 | |
| 1112 if (_collectorState < Sweeping) { | |
| 1113 // Mark the appropriate cards in the modUnionTable, so that | |
| 1114 // this object gets scanned before the sweep. If this is | |
| 1115 // not done, CMS generation references in the object might | |
| 1116 // not get marked. | |
| 1117 // For the case of arrays, which are otherwise precisely | |
| 1118 // marked, we need to dirty the entire array, not just its head. | |
| 1119 if (is_obj_array) { | |
| 1120 // The [par_]mark_range() method expects mr.end() below to | |
| 1121 // be aligned to the granularity of a bit's representation | |
| 1122 // in the heap. In the case of the MUT below, that's a | |
| 1123 // card size. | |
| 1124 MemRegion mr(start, | |
| 1125 (HeapWord*)round_to((intptr_t)(start + obj_size), | |
| 1126 CardTableModRefBS::card_size /* bytes */)); | |
| 1127 if (par) { | |
| 1128 _modUnionTable.par_mark_range(mr); | |
| 1129 } else { | |
| 1130 _modUnionTable.mark_range(mr); | |
| 1131 } | |
| 1132 } else { // not an obj array; we can just mark the head | |
| 1133 if (par) { | |
| 1134 _modUnionTable.par_mark(start); | |
| 1135 } else { | |
| 1136 _modUnionTable.mark(start); | |
| 1137 } | |
| 1138 } | |
| 1139 } | |
| 1140 } | |
| 1141 } | |
| 1142 | |
| 1143 static inline size_t percent_of_space(Space* space, HeapWord* addr) | |
| 1144 { | |
| 1145 size_t delta = pointer_delta(addr, space->bottom()); | |
| 1146 return (size_t)(delta * 100.0 / (space->capacity() / HeapWordSize)); | |
| 1147 } | |
| 1148 | |
| 1149 void CMSCollector::icms_update_allocation_limits() | |
| 1150 { | |
| 1151 Generation* gen0 = GenCollectedHeap::heap()->get_gen(0); | |
| 1152 EdenSpace* eden = gen0->as_DefNewGeneration()->eden(); | |
| 1153 | |
| 1154 const unsigned int duty_cycle = stats().icms_update_duty_cycle(); | |
| 1155 if (CMSTraceIncrementalPacing) { | |
| 1156 stats().print(); | |
| 1157 } | |
| 1158 | |
| 1159 assert(duty_cycle <= 100, "invalid duty cycle"); | |
| 1160 if (duty_cycle != 0) { | |
| 1161 // The duty_cycle is a percentage between 0 and 100; convert to words and | |
| 1162 // then compute the offset from the endpoints of the space. | |
| 1163 size_t free_words = eden->free() / HeapWordSize; | |
| 1164 double free_words_dbl = (double)free_words; | |
| 1165 size_t duty_cycle_words = (size_t)(free_words_dbl * duty_cycle / 100.0); | |
| 1166 size_t offset_words = (free_words - duty_cycle_words) / 2; | |
| 1167 | |
| 1168 _icms_start_limit = eden->top() + offset_words; | |
| 1169 _icms_stop_limit = eden->end() - offset_words; | |
| 1170 | |
| 1171 // The limits may be adjusted (shifted to the right) by | |
| 1172 // CMSIncrementalOffset, to allow the application more mutator time after a | |
| 1173 // young gen gc (when all mutators were stopped) and before CMS starts and | |
| 1174 // takes away one or more cpus. | |
| 1175 if (CMSIncrementalOffset != 0) { | |
| 1176 double adjustment_dbl = free_words_dbl * CMSIncrementalOffset / 100.0; | |
| 1177 size_t adjustment = (size_t)adjustment_dbl; | |
| 1178 HeapWord* tmp_stop = _icms_stop_limit + adjustment; | |
| 1179 if (tmp_stop > _icms_stop_limit && tmp_stop < eden->end()) { | |
| 1180 _icms_start_limit += adjustment; | |
| 1181 _icms_stop_limit = tmp_stop; | |
| 1182 } | |
| 1183 } | |
| 1184 } | |
| 1185 if (duty_cycle == 0 || (_icms_start_limit == _icms_stop_limit)) { | |
| 1186 _icms_start_limit = _icms_stop_limit = eden->end(); | |
| 1187 } | |
| 1188 | |
| 1189 // Install the new start limit. | |
| 1190 eden->set_soft_end(_icms_start_limit); | |
| 1191 | |
| 1192 if (CMSTraceIncrementalMode) { | |
| 1193 gclog_or_tty->print(" icms alloc limits: " | |
| 1194 PTR_FORMAT "," PTR_FORMAT | |
| 1195 " (" SIZE_FORMAT "%%," SIZE_FORMAT "%%) ", | |
| 1196 _icms_start_limit, _icms_stop_limit, | |
| 1197 percent_of_space(eden, _icms_start_limit), | |
| 1198 percent_of_space(eden, _icms_stop_limit)); | |
| 1199 if (Verbose) { | |
| 1200 gclog_or_tty->print("eden: "); | |
| 1201 eden->print_on(gclog_or_tty); | |
| 1202 } | |
| 1203 } | |
| 1204 } | |
| 1205 | |
| 1206 // Any changes here should try to maintain the invariant | |
| 1207 // that if this method is called with _icms_start_limit | |
| 1208 // and _icms_stop_limit both NULL, then it should return NULL | |
| 1209 // and not notify the icms thread. | |
| 1210 HeapWord* | |
| 1211 CMSCollector::allocation_limit_reached(Space* space, HeapWord* top, | |
| 1212 size_t word_size) | |
| 1213 { | |
| 1214 // A start_limit equal to end() means the duty cycle is 0, so treat that as a | |
| 1215 // nop. | |
| 1216 if (CMSIncrementalMode && _icms_start_limit != space->end()) { | |
| 1217 if (top <= _icms_start_limit) { | |
| 1218 if (CMSTraceIncrementalMode) { | |
| 1219 space->print_on(gclog_or_tty); | |
| 1220 gclog_or_tty->stamp(); | |
| 1221 gclog_or_tty->print_cr(" start limit top=" PTR_FORMAT | |
| 1222 ", new limit=" PTR_FORMAT | |
| 1223 " (" SIZE_FORMAT "%%)", | |
| 1224 top, _icms_stop_limit, | |
| 1225 percent_of_space(space, _icms_stop_limit)); | |
| 1226 } | |
| 1227 ConcurrentMarkSweepThread::start_icms(); | |
| 1228 assert(top < _icms_stop_limit, "Tautology"); | |
| 1229 if (word_size < pointer_delta(_icms_stop_limit, top)) { | |
| 1230 return _icms_stop_limit; | |
| 1231 } | |
| 1232 | |
| 1233 // The allocation will cross both the _start and _stop limits, so do the | |
| 1234 // stop notification also and return end(). | |
| 1235 if (CMSTraceIncrementalMode) { | |
| 1236 space->print_on(gclog_or_tty); | |
| 1237 gclog_or_tty->stamp(); | |
| 1238 gclog_or_tty->print_cr(" +stop limit top=" PTR_FORMAT | |
| 1239 ", new limit=" PTR_FORMAT | |
| 1240 " (" SIZE_FORMAT "%%)", | |
| 1241 top, space->end(), | |
| 1242 percent_of_space(space, space->end())); | |
| 1243 } | |
| 1244 ConcurrentMarkSweepThread::stop_icms(); | |
| 1245 return space->end(); | |
| 1246 } | |
| 1247 | |
| 1248 if (top <= _icms_stop_limit) { | |
| 1249 if (CMSTraceIncrementalMode) { | |
| 1250 space->print_on(gclog_or_tty); | |
| 1251 gclog_or_tty->stamp(); | |
| 1252 gclog_or_tty->print_cr(" stop limit top=" PTR_FORMAT | |
| 1253 ", new limit=" PTR_FORMAT | |
| 1254 " (" SIZE_FORMAT "%%)", | |
| 1255 top, space->end(), | |
| 1256 percent_of_space(space, space->end())); | |
| 1257 } | |
| 1258 ConcurrentMarkSweepThread::stop_icms(); | |
| 1259 return space->end(); | |
| 1260 } | |
| 1261 | |
| 1262 if (CMSTraceIncrementalMode) { | |
| 1263 space->print_on(gclog_or_tty); | |
| 1264 gclog_or_tty->stamp(); | |
| 1265 gclog_or_tty->print_cr(" end limit top=" PTR_FORMAT | |
| 1266 ", new limit=" PTR_FORMAT, | |
| 1267 top, NULL); | |
| 1268 } | |
| 1269 } | |
| 1270 | |
| 1271 return NULL; | |
| 1272 } | |
| 1273 | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
1274 oop ConcurrentMarkSweepGeneration::promote(oop obj, size_t obj_size) { |
| 0 | 1275 assert(obj_size == (size_t)obj->size(), "bad obj_size passed in"); |
| 1276 // allocate, copy and if necessary update promoinfo -- | |
| 1277 // delegate to underlying space. | |
| 1278 assert_lock_strong(freelistLock()); | |
| 1279 | |
| 1280 #ifndef PRODUCT | |
| 1281 if (Universe::heap()->promotion_should_fail()) { | |
| 1282 return NULL; | |
| 1283 } | |
| 1284 #endif // #ifndef PRODUCT | |
| 1285 | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
1286 oop res = _cmsSpace->promote(obj, obj_size); |
| 0 | 1287 if (res == NULL) { |
| 1288 // expand and retry | |
| 1289 size_t s = _cmsSpace->expansionSpaceRequired(obj_size); // HeapWords | |
| 1290 expand(s*HeapWordSize, MinHeapDeltaBytes, | |
| 1291 CMSExpansionCause::_satisfy_promotion); | |
| 1292 // Since there's currently no next generation, we don't try to promote | |
| 1293 // into a more senior generation. | |
| 1294 assert(next_gen() == NULL, "assumption, based upon which no attempt " | |
| 1295 "is made to pass on a possibly failing " | |
| 1296 "promotion to next generation"); | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
1297 res = _cmsSpace->promote(obj, obj_size); |
| 0 | 1298 } |
| 1299 if (res != NULL) { | |
| 1300 // See comment in allocate() about when objects should | |
| 1301 // be allocated live. | |
| 1302 assert(obj->is_oop(), "Will dereference klass pointer below"); | |
| 1303 collector()->promoted(false, // Not parallel | |
| 1304 (HeapWord*)res, obj->is_objArray(), obj_size); | |
| 1305 // promotion counters | |
| 1306 NOT_PRODUCT( | |
| 1307 _numObjectsPromoted++; | |
| 1308 _numWordsPromoted += | |
| 1309 (int)(CompactibleFreeListSpace::adjustObjectSize(obj->size())); | |
| 1310 ) | |
| 1311 } | |
| 1312 return res; | |
| 1313 } | |
| 1314 | |
| 1315 | |
| 1316 HeapWord* | |
| 1317 ConcurrentMarkSweepGeneration::allocation_limit_reached(Space* space, | |
| 1318 HeapWord* top, | |
| 1319 size_t word_sz) | |
| 1320 { | |
| 1321 return collector()->allocation_limit_reached(space, top, word_sz); | |
| 1322 } | |
| 1323 | |
|
1716
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1324 // IMPORTANT: Notes on object size recognition in CMS. |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1325 // --------------------------------------------------- |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1326 // A block of storage in the CMS generation is always in |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1327 // one of three states. A free block (FREE), an allocated |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1328 // object (OBJECT) whose size() method reports the correct size, |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1329 // and an intermediate state (TRANSIENT) in which its size cannot |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1330 // be accurately determined. |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1331 // STATE IDENTIFICATION: (32 bit and 64 bit w/o COOPS) |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1332 // ----------------------------------------------------- |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1333 // FREE: klass_word & 1 == 1; mark_word holds block size |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1334 // |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1335 // OBJECT: klass_word installed; klass_word != 0 && klass_word & 0 == 0; |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1336 // obj->size() computes correct size |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1337 // [Perm Gen objects needs to be "parsable" before they can be navigated] |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1338 // |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1339 // TRANSIENT: klass_word == 0; size is indeterminate until we become an OBJECT |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1340 // |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1341 // STATE IDENTIFICATION: (64 bit+COOPS) |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1342 // ------------------------------------ |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1343 // FREE: mark_word & CMS_FREE_BIT == 1; mark_word & ~CMS_FREE_BIT gives block_size |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1344 // |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1345 // OBJECT: klass_word installed; klass_word != 0; |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1346 // obj->size() computes correct size |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1347 // [Perm Gen comment above continues to hold] |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1348 // |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1349 // TRANSIENT: klass_word == 0; size is indeterminate until we become an OBJECT |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1350 // |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1351 // |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1352 // STATE TRANSITION DIAGRAM |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1353 // |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1354 // mut / parnew mut / parnew |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1355 // FREE --------------------> TRANSIENT ---------------------> OBJECT --| |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1356 // ^ | |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1357 // |------------------------ DEAD <------------------------------------| |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1358 // sweep mut |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1359 // |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1360 // While a block is in TRANSIENT state its size cannot be determined |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1361 // so readers will either need to come back later or stall until |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1362 // the size can be determined. Note that for the case of direct |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1363 // allocation, P-bits, when available, may be used to determine the |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1364 // size of an object that may not yet have been initialized. |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1365 |
| 0 | 1366 // Things to support parallel young-gen collection. |
| 1367 oop | |
| 1368 ConcurrentMarkSweepGeneration::par_promote(int thread_num, | |
| 1369 oop old, markOop m, | |
| 1370 size_t word_sz) { | |
| 1371 #ifndef PRODUCT | |
| 1372 if (Universe::heap()->promotion_should_fail()) { | |
| 1373 return NULL; | |
| 1374 } | |
| 1375 #endif // #ifndef PRODUCT | |
| 1376 | |
| 1377 CMSParGCThreadState* ps = _par_gc_thread_states[thread_num]; | |
| 1378 PromotionInfo* promoInfo = &ps->promo; | |
| 1379 // if we are tracking promotions, then first ensure space for | |
| 1380 // promotion (including spooling space for saving header if necessary). | |
| 1381 // then allocate and copy, then track promoted info if needed. | |
| 1382 // When tracking (see PromotionInfo::track()), the mark word may | |
| 1383 // be displaced and in this case restoration of the mark word | |
| 1384 // occurs in the (oop_since_save_marks_)iterate phase. | |
| 1385 if (promoInfo->tracking() && !promoInfo->ensure_spooling_space()) { | |
| 1386 // Out of space for allocating spooling buffers; | |
| 1387 // try expanding and allocating spooling buffers. | |
| 1388 if (!expand_and_ensure_spooling_space(promoInfo)) { | |
| 1389 return NULL; | |
| 1390 } | |
| 1391 } | |
| 1392 assert(promoInfo->has_spooling_space(), "Control point invariant"); | |
|
1716
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1393 const size_t alloc_sz = CompactibleFreeListSpace::adjustObjectSize(word_sz); |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1394 HeapWord* obj_ptr = ps->lab.alloc(alloc_sz); |
| 0 | 1395 if (obj_ptr == NULL) { |
|
1716
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1396 obj_ptr = expand_and_par_lab_allocate(ps, alloc_sz); |
| 0 | 1397 if (obj_ptr == NULL) { |
| 1398 return NULL; | |
| 1399 } | |
| 1400 } | |
| 1401 oop obj = oop(obj_ptr); | |
|
1716
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1402 OrderAccess::storestore(); |
| 187 | 1403 assert(obj->klass_or_null() == NULL, "Object should be uninitialized here."); |
|
1716
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1404 assert(!((FreeChunk*)obj_ptr)->isFree(), "Error, block will look free but show wrong size"); |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1405 // IMPORTANT: See note on object initialization for CMS above. |
| 0 | 1406 // Otherwise, copy the object. Here we must be careful to insert the |
| 1407 // klass pointer last, since this marks the block as an allocated object. | |
| 187 | 1408 // Except with compressed oops it's the mark word. |
| 0 | 1409 HeapWord* old_ptr = (HeapWord*)old; |
|
1716
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1410 // Restore the mark word copied above. |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1411 obj->set_mark(m); |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1412 assert(obj->klass_or_null() == NULL, "Object should be uninitialized here."); |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1413 assert(!((FreeChunk*)obj_ptr)->isFree(), "Error, block will look free but show wrong size"); |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1414 OrderAccess::storestore(); |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1415 |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1416 if (UseCompressedOops) { |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1417 // Copy gap missed by (aligned) header size calculation below |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1418 obj->set_klass_gap(old->klass_gap()); |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1419 } |
| 0 | 1420 if (word_sz > (size_t)oopDesc::header_size()) { |
| 1421 Copy::aligned_disjoint_words(old_ptr + oopDesc::header_size(), | |
| 1422 obj_ptr + oopDesc::header_size(), | |
| 1423 word_sz - oopDesc::header_size()); | |
| 1424 } | |
| 187 | 1425 |
| 0 | 1426 // Now we can track the promoted object, if necessary. We take care |
| 1521 | 1427 // to delay the transition from uninitialized to full object |
| 0 | 1428 // (i.e., insertion of klass pointer) until after, so that it |
| 1429 // atomically becomes a promoted object. | |
| 1430 if (promoInfo->tracking()) { | |
| 1431 promoInfo->track((PromotedObject*)obj, old->klass()); | |
| 1432 } | |
|
1716
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1433 assert(obj->klass_or_null() == NULL, "Object should be uninitialized here."); |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1434 assert(!((FreeChunk*)obj_ptr)->isFree(), "Error, block will look free but show wrong size"); |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1435 assert(old->is_oop(), "Will use and dereference old klass ptr below"); |
| 187 | 1436 |
| 1437 // Finally, install the klass pointer (this should be volatile). | |
|
1716
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1438 OrderAccess::storestore(); |
| 0 | 1439 obj->set_klass(old->klass()); |
|
1716
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1440 // We should now be able to calculate the right size for this object |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1441 assert(obj->is_oop() && obj->size() == (int)word_sz, "Error, incorrect size computed for promoted object"); |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1442 |
| 0 | 1443 collector()->promoted(true, // parallel |
| 1444 obj_ptr, old->is_objArray(), word_sz); | |
| 1445 | |
| 1446 NOT_PRODUCT( | |
|
1716
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1447 Atomic::inc_ptr(&_numObjectsPromoted); |
|
be3f9c242c9d
6948538: CMS: BOT walkers can fall into object allocation and initialization cracks
ysr
parents:
1704
diff
changeset
|
1448 Atomic::add_ptr(alloc_sz, &_numWordsPromoted); |
| 0 | 1449 ) |
| 1450 | |
| 1451 return obj; | |
| 1452 } | |
| 1453 | |
| 1454 void | |
| 1455 ConcurrentMarkSweepGeneration:: | |
| 1456 par_promote_alloc_undo(int thread_num, | |
| 1457 HeapWord* obj, size_t word_sz) { | |
| 1458 // CMS does not support promotion undo. | |
| 1459 ShouldNotReachHere(); | |
| 1460 } | |
| 1461 | |
| 1462 void | |
| 1463 ConcurrentMarkSweepGeneration:: | |
| 1464 par_promote_alloc_done(int thread_num) { | |
| 1465 CMSParGCThreadState* ps = _par_gc_thread_states[thread_num]; | |
|
1145
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
1466 ps->lab.retire(thread_num); |
| 0 | 1467 } |
| 1468 | |
| 1469 void | |
| 1470 ConcurrentMarkSweepGeneration:: | |
| 1471 par_oop_since_save_marks_iterate_done(int thread_num) { | |
| 1472 CMSParGCThreadState* ps = _par_gc_thread_states[thread_num]; | |
| 1473 ParScanWithoutBarrierClosure* dummy_cl = NULL; | |
| 1474 ps->promo.promoted_oops_iterate_nv(dummy_cl); | |
| 1475 } | |
| 1476 | |
| 1477 // XXXPERM | |
| 1478 bool ConcurrentMarkSweepGeneration::should_collect(bool full, | |
| 1479 size_t size, | |
| 1480 bool tlab) | |
| 1481 { | |
| 1482 // We allow a STW collection only if a full | |
| 1483 // collection was requested. | |
| 1484 return full || should_allocate(size, tlab); // FIX ME !!! | |
| 1485 // This and promotion failure handling are connected at the | |
| 1486 // hip and should be fixed by untying them. | |
| 1487 } | |
| 1488 | |
| 1489 bool CMSCollector::shouldConcurrentCollect() { | |
| 1490 if (_full_gc_requested) { | |
| 1491 if (Verbose && PrintGCDetails) { | |
| 1492 gclog_or_tty->print_cr("CMSCollector: collect because of explicit " | |
|
1520
bb843ebc7c55
6919638: CMS: ExplicitGCInvokesConcurrent misinteracts with gc locker
ysr
parents:
1518
diff
changeset
|
1493 " gc request (or gc_locker)"); |
| 0 | 1494 } |
| 1495 return true; | |
| 1496 } | |
| 1497 | |
| 1498 // For debugging purposes, change the type of collection. | |
| 1499 // If the rotation is not on the concurrent collection | |
| 1500 // type, don't start a concurrent collection. | |
| 1501 NOT_PRODUCT( | |
| 1502 if (RotateCMSCollectionTypes && | |
| 1503 (_cmsGen->debug_collection_type() != | |
| 1504 ConcurrentMarkSweepGeneration::Concurrent_collection_type)) { | |
| 1505 assert(_cmsGen->debug_collection_type() != | |
| 1506 ConcurrentMarkSweepGeneration::Unknown_collection_type, | |
| 1507 "Bad cms collection type"); | |
| 1508 return false; | |
| 1509 } | |
| 1510 ) | |
| 1511 | |
| 1512 FreelistLocker x(this); | |
| 1513 // ------------------------------------------------------------------ | |
| 1514 // Print out lots of information which affects the initiation of | |
| 1515 // a collection. | |
| 1516 if (PrintCMSInitiationStatistics && stats().valid()) { | |
| 1517 gclog_or_tty->print("CMSCollector shouldConcurrentCollect: "); | |
| 1518 gclog_or_tty->stamp(); | |
| 1519 gclog_or_tty->print_cr(""); | |
| 1520 stats().print_on(gclog_or_tty); | |
| 1521 gclog_or_tty->print_cr("time_until_cms_gen_full %3.7f", | |
| 1522 stats().time_until_cms_gen_full()); | |
| 1523 gclog_or_tty->print_cr("free="SIZE_FORMAT, _cmsGen->free()); | |
| 1524 gclog_or_tty->print_cr("contiguous_available="SIZE_FORMAT, | |
| 1525 _cmsGen->contiguous_available()); | |
| 1526 gclog_or_tty->print_cr("promotion_rate=%g", stats().promotion_rate()); | |
| 1527 gclog_or_tty->print_cr("cms_allocation_rate=%g", stats().cms_allocation_rate()); | |
| 1528 gclog_or_tty->print_cr("occupancy=%3.7f", _cmsGen->occupancy()); | |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1529 gclog_or_tty->print_cr("initiatingOccupancy=%3.7f", _cmsGen->initiating_occupancy()); |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1530 gclog_or_tty->print_cr("initiatingPermOccupancy=%3.7f", _permGen->initiating_occupancy()); |
| 0 | 1531 } |
| 1532 // ------------------------------------------------------------------ | |
| 1533 | |
| 1534 // If the estimated time to complete a cms collection (cms_duration()) | |
| 1535 // is less than the estimated time remaining until the cms generation | |
| 1536 // is full, start a collection. | |
| 1537 if (!UseCMSInitiatingOccupancyOnly) { | |
| 1538 if (stats().valid()) { | |
| 1539 if (stats().time_until_cms_start() == 0.0) { | |
| 1540 return true; | |
| 1541 } | |
| 1542 } else { | |
| 1543 // We want to conservatively collect somewhat early in order | |
| 1544 // to try and "bootstrap" our CMS/promotion statistics; | |
| 1545 // this branch will not fire after the first successful CMS | |
| 1546 // collection because the stats should then be valid. | |
| 1547 if (_cmsGen->occupancy() >= _bootstrap_occupancy) { | |
| 1548 if (Verbose && PrintGCDetails) { | |
| 1549 gclog_or_tty->print_cr( | |
| 1550 " CMSCollector: collect for bootstrapping statistics:" | |
| 1551 " occupancy = %f, boot occupancy = %f", _cmsGen->occupancy(), | |
| 1552 _bootstrap_occupancy); | |
| 1553 } | |
| 1554 return true; | |
| 1555 } | |
| 1556 } | |
| 1557 } | |
| 1558 | |
| 1559 // Otherwise, we start a collection cycle if either the perm gen or | |
| 1560 // old gen want a collection cycle started. Each may use | |
| 1561 // an appropriate criterion for making this decision. | |
| 1562 // XXX We need to make sure that the gen expansion | |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1563 // criterion dovetails well with this. XXX NEED TO FIX THIS |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1564 if (_cmsGen->should_concurrent_collect()) { |
| 0 | 1565 if (Verbose && PrintGCDetails) { |
| 1566 gclog_or_tty->print_cr("CMS old gen initiated"); | |
| 1567 } | |
| 1568 return true; | |
| 1569 } | |
| 1570 | |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1571 // We start a collection if we believe an incremental collection may fail; |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1572 // this is not likely to be productive in practice because it's probably too |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1573 // late anyway. |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1574 GenCollectedHeap* gch = GenCollectedHeap::heap(); |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1575 assert(gch->collector_policy()->is_two_generation_policy(), |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1576 "You may want to check the correctness of the following"); |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1577 if (gch->incremental_collection_will_fail()) { |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1578 if (PrintGCDetails && Verbose) { |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1579 gclog_or_tty->print("CMSCollector: collect because incremental collection will fail "); |
| 0 | 1580 } |
| 1581 return true; | |
| 1582 } | |
| 1583 | |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1584 if (CMSClassUnloadingEnabled && _permGen->should_concurrent_collect()) { |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1585 bool res = update_should_unload_classes(); |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1586 if (res) { |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1587 if (Verbose && PrintGCDetails) { |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1588 gclog_or_tty->print_cr("CMS perm gen initiated"); |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1589 } |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1590 return true; |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1591 } |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1592 } |
| 0 | 1593 return false; |
| 1594 } | |
| 1595 | |
| 1596 // Clear _expansion_cause fields of constituent generations | |
| 1597 void CMSCollector::clear_expansion_cause() { | |
| 1598 _cmsGen->clear_expansion_cause(); | |
| 1599 _permGen->clear_expansion_cause(); | |
| 1600 } | |
| 1601 | |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1602 // We should be conservative in starting a collection cycle. To |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1603 // start too eagerly runs the risk of collecting too often in the |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1604 // extreme. To collect too rarely falls back on full collections, |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1605 // which works, even if not optimum in terms of concurrent work. |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1606 // As a work around for too eagerly collecting, use the flag |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1607 // UseCMSInitiatingOccupancyOnly. This also has the advantage of |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1608 // giving the user an easily understandable way of controlling the |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1609 // collections. |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1610 // We want to start a new collection cycle if any of the following |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1611 // conditions hold: |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1612 // . our current occupancy exceeds the configured initiating occupancy |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1613 // for this generation, or |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1614 // . we recently needed to expand this space and have not, since that |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1615 // expansion, done a collection of this generation, or |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1616 // . the underlying space believes that it may be a good idea to initiate |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1617 // a concurrent collection (this may be based on criteria such as the |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1618 // following: the space uses linear allocation and linear allocation is |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1619 // going to fail, or there is believed to be excessive fragmentation in |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1620 // the generation, etc... or ... |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1621 // [.(currently done by CMSCollector::shouldConcurrentCollect() only for |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1622 // the case of the old generation, not the perm generation; see CR 6543076): |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1623 // we may be approaching a point at which allocation requests may fail because |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1624 // we will be out of sufficient free space given allocation rate estimates.] |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1625 bool ConcurrentMarkSweepGeneration::should_concurrent_collect() const { |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1626 |
| 0 | 1627 assert_lock_strong(freelistLock()); |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1628 if (occupancy() > initiating_occupancy()) { |
| 0 | 1629 if (PrintGCDetails && Verbose) { |
| 1630 gclog_or_tty->print(" %s: collect because of occupancy %f / %f ", | |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1631 short_name(), occupancy(), initiating_occupancy()); |
| 0 | 1632 } |
| 1633 return true; | |
| 1634 } | |
| 1635 if (UseCMSInitiatingOccupancyOnly) { | |
| 1636 return false; | |
| 1637 } | |
| 1638 if (expansion_cause() == CMSExpansionCause::_satisfy_allocation) { | |
| 1639 if (PrintGCDetails && Verbose) { | |
| 1640 gclog_or_tty->print(" %s: collect because expanded for allocation ", | |
| 1641 short_name()); | |
| 1642 } | |
| 1643 return true; | |
| 1644 } | |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1645 if (_cmsSpace->should_concurrent_collect()) { |
| 0 | 1646 if (PrintGCDetails && Verbose) { |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
1647 gclog_or_tty->print(" %s: collect because cmsSpace says so ", |
| 0 | 1648 short_name()); |
| 1649 } | |
| 1650 return true; | |
| 1651 } | |
| 1652 return false; | |
| 1653 } | |
| 1654 | |
| 1655 void ConcurrentMarkSweepGeneration::collect(bool full, | |
| 1656 bool clear_all_soft_refs, | |
| 1657 size_t size, | |
| 1658 bool tlab) | |
| 1659 { | |
| 1660 collector()->collect(full, clear_all_soft_refs, size, tlab); | |
| 1661 } | |
| 1662 | |
| 1663 void CMSCollector::collect(bool full, | |
| 1664 bool clear_all_soft_refs, | |
| 1665 size_t size, | |
| 1666 bool tlab) | |
| 1667 { | |
| 1668 if (!UseCMSCollectionPassing && _collectorState > Idling) { | |
| 1669 // For debugging purposes skip the collection if the state | |
| 1670 // is not currently idle | |
| 1671 if (TraceCMSState) { | |
| 1672 gclog_or_tty->print_cr("Thread " INTPTR_FORMAT " skipped full:%d CMS state %d", | |
| 1673 Thread::current(), full, _collectorState); | |
| 1674 } | |
| 1675 return; | |
| 1676 } | |
| 1677 | |
| 1678 // The following "if" branch is present for defensive reasons. | |
| 1679 // In the current uses of this interface, it can be replaced with: | |
| 1680 // assert(!GC_locker.is_active(), "Can't be called otherwise"); | |
| 1681 // But I am not placing that assert here to allow future | |
| 1682 // generality in invoking this interface. | |
| 1683 if (GC_locker::is_active()) { | |
| 1684 // A consistency test for GC_locker | |
| 1685 assert(GC_locker::needs_gc(), "Should have been set already"); | |
| 1686 // Skip this foreground collection, instead | |
| 1687 // expanding the heap if necessary. | |
| 1688 // Need the free list locks for the call to free() in compute_new_size() | |
| 1689 compute_new_size(); | |
| 1690 return; | |
| 1691 } | |
| 1692 acquire_control_and_collect(full, clear_all_soft_refs); | |
| 1693 _full_gcs_since_conc_gc++; | |
| 1694 | |
| 1695 } | |
| 1696 | |
| 1697 void CMSCollector::request_full_gc(unsigned int full_gc_count) { | |
| 1698 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 1699 unsigned int gc_count = gch->total_full_collections(); | |
| 1700 if (gc_count == full_gc_count) { | |
| 1701 MutexLockerEx y(CGC_lock, Mutex::_no_safepoint_check_flag); | |
| 1702 _full_gc_requested = true; | |
| 1703 CGC_lock->notify(); // nudge CMS thread | |
| 1704 } | |
| 1705 } | |
| 1706 | |
| 1707 | |
| 1708 // The foreground and background collectors need to coordinate in order | |
| 1709 // to make sure that they do not mutually interfere with CMS collections. | |
| 1710 // When a background collection is active, | |
| 1711 // the foreground collector may need to take over (preempt) and | |
| 1712 // synchronously complete an ongoing collection. Depending on the | |
| 1713 // frequency of the background collections and the heap usage | |
| 1714 // of the application, this preemption can be seldom or frequent. | |
| 1715 // There are only certain | |
| 1716 // points in the background collection that the "collection-baton" | |
| 1717 // can be passed to the foreground collector. | |
| 1718 // | |
| 1719 // The foreground collector will wait for the baton before | |
| 1720 // starting any part of the collection. The foreground collector | |
| 1721 // will only wait at one location. | |
| 1722 // | |
| 1723 // The background collector will yield the baton before starting a new | |
| 1724 // phase of the collection (e.g., before initial marking, marking from roots, | |
| 1725 // precleaning, final re-mark, sweep etc.) This is normally done at the head | |
| 1726 // of the loop which switches the phases. The background collector does some | |
| 1727 // of the phases (initial mark, final re-mark) with the world stopped. | |
| 1728 // Because of locking involved in stopping the world, | |
| 1729 // the foreground collector should not block waiting for the background | |
| 1730 // collector when it is doing a stop-the-world phase. The background | |
| 1731 // collector will yield the baton at an additional point just before | |
| 1732 // it enters a stop-the-world phase. Once the world is stopped, the | |
| 1733 // background collector checks the phase of the collection. If the | |
| 1734 // phase has not changed, it proceeds with the collection. If the | |
| 1735 // phase has changed, it skips that phase of the collection. See | |
| 1736 // the comments on the use of the Heap_lock in collect_in_background(). | |
| 1737 // | |
| 1738 // Variable used in baton passing. | |
| 1739 // _foregroundGCIsActive - Set to true by the foreground collector when | |
| 1740 // it wants the baton. The foreground clears it when it has finished | |
| 1741 // the collection. | |
| 1742 // _foregroundGCShouldWait - Set to true by the background collector | |
| 1743 // when it is running. The foreground collector waits while | |
| 1744 // _foregroundGCShouldWait is true. | |
| 1745 // CGC_lock - monitor used to protect access to the above variables | |
| 1746 // and to notify the foreground and background collectors. | |
| 1747 // _collectorState - current state of the CMS collection. | |
| 1748 // | |
| 1749 // The foreground collector | |
| 1750 // acquires the CGC_lock | |
| 1751 // sets _foregroundGCIsActive | |
| 1752 // waits on the CGC_lock for _foregroundGCShouldWait to be false | |
| 1753 // various locks acquired in preparation for the collection | |
| 1754 // are released so as not to block the background collector | |
| 1755 // that is in the midst of a collection | |
| 1756 // proceeds with the collection | |
| 1757 // clears _foregroundGCIsActive | |
| 1758 // returns | |
| 1759 // | |
| 1760 // The background collector in a loop iterating on the phases of the | |
| 1761 // collection | |
| 1762 // acquires the CGC_lock | |
| 1763 // sets _foregroundGCShouldWait | |
| 1764 // if _foregroundGCIsActive is set | |
| 1765 // clears _foregroundGCShouldWait, notifies _CGC_lock | |
| 1766 // waits on _CGC_lock for _foregroundGCIsActive to become false | |
| 1767 // and exits the loop. | |
| 1768 // otherwise | |
| 1769 // proceed with that phase of the collection | |
| 1770 // if the phase is a stop-the-world phase, | |
| 1771 // yield the baton once more just before enqueueing | |
| 1772 // the stop-world CMS operation (executed by the VM thread). | |
| 1773 // returns after all phases of the collection are done | |
| 1774 // | |
| 1775 | |
| 1776 void CMSCollector::acquire_control_and_collect(bool full, | |
| 1777 bool clear_all_soft_refs) { | |
| 1778 assert(SafepointSynchronize::is_at_safepoint(), "should be at safepoint"); | |
| 1779 assert(!Thread::current()->is_ConcurrentGC_thread(), | |
| 1780 "shouldn't try to acquire control from self!"); | |
| 1781 | |
| 1782 // Start the protocol for acquiring control of the | |
| 1783 // collection from the background collector (aka CMS thread). | |
| 1784 assert(ConcurrentMarkSweepThread::vm_thread_has_cms_token(), | |
| 1785 "VM thread should have CMS token"); | |
| 1786 // Remember the possibly interrupted state of an ongoing | |
| 1787 // concurrent collection | |
| 1788 CollectorState first_state = _collectorState; | |
| 1789 | |
| 1790 // Signal to a possibly ongoing concurrent collection that | |
| 1791 // we want to do a foreground collection. | |
| 1792 _foregroundGCIsActive = true; | |
| 1793 | |
| 1794 // Disable incremental mode during a foreground collection. | |
| 1795 ICMSDisabler icms_disabler; | |
| 1796 | |
| 1797 // release locks and wait for a notify from the background collector | |
| 1798 // releasing the locks in only necessary for phases which | |
| 1799 // do yields to improve the granularity of the collection. | |
| 1800 assert_lock_strong(bitMapLock()); | |
| 1801 // We need to lock the Free list lock for the space that we are | |
| 1802 // currently collecting. | |
| 1803 assert(haveFreelistLocks(), "Must be holding free list locks"); | |
| 1804 bitMapLock()->unlock(); | |
| 1805 releaseFreelistLocks(); | |
| 1806 { | |
| 1807 MutexLockerEx x(CGC_lock, Mutex::_no_safepoint_check_flag); | |
| 1808 if (_foregroundGCShouldWait) { | |
| 1809 // We are going to be waiting for action for the CMS thread; | |
| 1810 // it had better not be gone (for instance at shutdown)! | |
| 1811 assert(ConcurrentMarkSweepThread::cmst() != NULL, | |
| 1812 "CMS thread must be running"); | |
| 1813 // Wait here until the background collector gives us the go-ahead | |
| 1814 ConcurrentMarkSweepThread::clear_CMS_flag( | |
| 1815 ConcurrentMarkSweepThread::CMS_vm_has_token); // release token | |
| 1816 // Get a possibly blocked CMS thread going: | |
| 1817 // Note that we set _foregroundGCIsActive true above, | |
| 1818 // without protection of the CGC_lock. | |
| 1819 CGC_lock->notify(); | |
| 1820 assert(!ConcurrentMarkSweepThread::vm_thread_wants_cms_token(), | |
| 1821 "Possible deadlock"); | |
| 1822 while (_foregroundGCShouldWait) { | |
| 1823 // wait for notification | |
| 1824 CGC_lock->wait(Mutex::_no_safepoint_check_flag); | |
| 1825 // Possibility of delay/starvation here, since CMS token does | |
| 1826 // not know to give priority to VM thread? Actually, i think | |
| 1827 // there wouldn't be any delay/starvation, but the proof of | |
| 1828 // that "fact" (?) appears non-trivial. XXX 20011219YSR | |
| 1829 } | |
| 1830 ConcurrentMarkSweepThread::set_CMS_flag( | |
| 1831 ConcurrentMarkSweepThread::CMS_vm_has_token); | |
| 1832 } | |
| 1833 } | |
| 1834 // The CMS_token is already held. Get back the other locks. | |
| 1835 assert(ConcurrentMarkSweepThread::vm_thread_has_cms_token(), | |
| 1836 "VM thread should have CMS token"); | |
| 1837 getFreelistLocks(); | |
| 1838 bitMapLock()->lock_without_safepoint_check(); | |
| 1839 if (TraceCMSState) { | |
| 1840 gclog_or_tty->print_cr("CMS foreground collector has asked for control " | |
| 1841 INTPTR_FORMAT " with first state %d", Thread::current(), first_state); | |
| 1842 gclog_or_tty->print_cr(" gets control with state %d", _collectorState); | |
| 1843 } | |
| 1844 | |
| 1845 // Check if we need to do a compaction, or if not, whether | |
| 1846 // we need to start the mark-sweep from scratch. | |
| 1847 bool should_compact = false; | |
| 1848 bool should_start_over = false; | |
| 1849 decide_foreground_collection_type(clear_all_soft_refs, | |
| 1850 &should_compact, &should_start_over); | |
| 1851 | |
| 1852 NOT_PRODUCT( | |
| 1853 if (RotateCMSCollectionTypes) { | |
| 1854 if (_cmsGen->debug_collection_type() == | |
| 1855 ConcurrentMarkSweepGeneration::MSC_foreground_collection_type) { | |
| 1856 should_compact = true; | |
| 1857 } else if (_cmsGen->debug_collection_type() == | |
| 1858 ConcurrentMarkSweepGeneration::MS_foreground_collection_type) { | |
| 1859 should_compact = false; | |
| 1860 } | |
| 1861 } | |
| 1862 ) | |
| 1863 | |
| 1864 if (PrintGCDetails && first_state > Idling) { | |
| 1865 GCCause::Cause cause = GenCollectedHeap::heap()->gc_cause(); | |
| 1866 if (GCCause::is_user_requested_gc(cause) || | |
| 1867 GCCause::is_serviceability_requested_gc(cause)) { | |
| 1868 gclog_or_tty->print(" (concurrent mode interrupted)"); | |
| 1869 } else { | |
| 1870 gclog_or_tty->print(" (concurrent mode failure)"); | |
| 1871 } | |
| 1872 } | |
| 1873 | |
| 1874 if (should_compact) { | |
| 1875 // If the collection is being acquired from the background | |
| 1876 // collector, there may be references on the discovered | |
| 1877 // references lists that have NULL referents (being those | |
| 1878 // that were concurrently cleared by a mutator) or | |
| 1879 // that are no longer active (having been enqueued concurrently | |
| 1880 // by the mutator). | |
| 1881 // Scrub the list of those references because Mark-Sweep-Compact | |
| 1882 // code assumes referents are not NULL and that all discovered | |
| 1883 // Reference objects are active. | |
| 1884 ref_processor()->clean_up_discovered_references(); | |
| 1885 | |
| 1886 do_compaction_work(clear_all_soft_refs); | |
| 1887 | |
| 1888 // Has the GC time limit been exceeded? | |
|
1387
0bfd3fb24150
6858496: Clear all SoftReferences before an out-of-memory due to GC overhead limit.
jmasa
parents:
1289
diff
changeset
|
1889 DefNewGeneration* young_gen = _young_gen->as_DefNewGeneration(); |
|
0bfd3fb24150
6858496: Clear all SoftReferences before an out-of-memory due to GC overhead limit.
jmasa
parents:
1289
diff
changeset
|
1890 size_t max_eden_size = young_gen->max_capacity() - |
|
0bfd3fb24150
6858496: Clear all SoftReferences before an out-of-memory due to GC overhead limit.
jmasa
parents:
1289
diff
changeset
|
1891 young_gen->to()->capacity() - |
|
0bfd3fb24150
6858496: Clear all SoftReferences before an out-of-memory due to GC overhead limit.
jmasa
parents:
1289
diff
changeset
|
1892 young_gen->from()->capacity(); |
|
0bfd3fb24150
6858496: Clear all SoftReferences before an out-of-memory due to GC overhead limit.
jmasa
parents:
1289
diff
changeset
|
1893 GenCollectedHeap* gch = GenCollectedHeap::heap(); |
|
0bfd3fb24150
6858496: Clear all SoftReferences before an out-of-memory due to GC overhead limit.
jmasa
parents:
1289
diff
changeset
|
1894 GCCause::Cause gc_cause = gch->gc_cause(); |
|
0bfd3fb24150
6858496: Clear all SoftReferences before an out-of-memory due to GC overhead limit.
jmasa
parents:
1289
diff
changeset
|
1895 size_policy()->check_gc_overhead_limit(_young_gen->used(), |
|
0bfd3fb24150
6858496: Clear all SoftReferences before an out-of-memory due to GC overhead limit.
jmasa
parents:
1289
diff
changeset
|
1896 young_gen->eden()->used(), |
|
0bfd3fb24150
6858496: Clear all SoftReferences before an out-of-memory due to GC overhead limit.
jmasa
parents:
1289
diff
changeset
|
1897 _cmsGen->max_capacity(), |
|
0bfd3fb24150
6858496: Clear all SoftReferences before an out-of-memory due to GC overhead limit.
jmasa
parents:
1289
diff
changeset
|
1898 max_eden_size, |
|
0bfd3fb24150
6858496: Clear all SoftReferences before an out-of-memory due to GC overhead limit.
jmasa
parents:
1289
diff
changeset
|
1899 full, |
|
0bfd3fb24150
6858496: Clear all SoftReferences before an out-of-memory due to GC overhead limit.
jmasa
parents:
1289
diff
changeset
|
1900 gc_cause, |
|
0bfd3fb24150
6858496: Clear all SoftReferences before an out-of-memory due to GC overhead limit.
jmasa
parents:
1289
diff
changeset
|
1901 gch->collector_policy()); |
| 0 | 1902 } else { |
| 1903 do_mark_sweep_work(clear_all_soft_refs, first_state, | |
| 1904 should_start_over); | |
| 1905 } | |
| 1906 // Reset the expansion cause, now that we just completed | |
| 1907 // a collection cycle. | |
| 1908 clear_expansion_cause(); | |
| 1909 _foregroundGCIsActive = false; | |
| 1910 return; | |
| 1911 } | |
| 1912 | |
| 1913 // Resize the perm generation and the tenured generation | |
| 1914 // after obtaining the free list locks for the | |
| 1915 // two generations. | |
| 1916 void CMSCollector::compute_new_size() { | |
| 1917 assert_locked_or_safepoint(Heap_lock); | |
| 1918 FreelistLocker z(this); | |
| 1919 _permGen->compute_new_size(); | |
| 1920 _cmsGen->compute_new_size(); | |
| 1921 } | |
| 1922 | |
| 1923 // A work method used by foreground collection to determine | |
| 1924 // what type of collection (compacting or not, continuing or fresh) | |
| 1925 // it should do. | |
| 1926 // NOTE: the intent is to make UseCMSCompactAtFullCollection | |
| 1927 // and CMSCompactWhenClearAllSoftRefs the default in the future | |
| 1928 // and do away with the flags after a suitable period. | |
| 1929 void CMSCollector::decide_foreground_collection_type( | |
| 1930 bool clear_all_soft_refs, bool* should_compact, | |
| 1931 bool* should_start_over) { | |
| 1932 // Normally, we'll compact only if the UseCMSCompactAtFullCollection | |
| 1933 // flag is set, and we have either requested a System.gc() or | |
| 1934 // the number of full gc's since the last concurrent cycle | |
| 1935 // has exceeded the threshold set by CMSFullGCsBeforeCompaction, | |
| 1936 // or if an incremental collection has failed | |
| 1937 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 1938 assert(gch->collector_policy()->is_two_generation_policy(), | |
| 1939 "You may want to check the correctness of the following"); | |
| 1940 // Inform cms gen if this was due to partial collection failing. | |
| 1941 // The CMS gen may use this fact to determine its expansion policy. | |
| 1942 if (gch->incremental_collection_will_fail()) { | |
| 1943 assert(!_cmsGen->incremental_collection_failed(), | |
| 1944 "Should have been noticed, reacted to and cleared"); | |
| 1945 _cmsGen->set_incremental_collection_failed(); | |
| 1946 } | |
| 1947 *should_compact = | |
| 1948 UseCMSCompactAtFullCollection && | |
| 1949 ((_full_gcs_since_conc_gc >= CMSFullGCsBeforeCompaction) || | |
| 1950 GCCause::is_user_requested_gc(gch->gc_cause()) || | |
| 1951 gch->incremental_collection_will_fail()); | |
| 1952 *should_start_over = false; | |
| 1953 if (clear_all_soft_refs && !*should_compact) { | |
| 1954 // We are about to do a last ditch collection attempt | |
| 1955 // so it would normally make sense to do a compaction | |
| 1956 // to reclaim as much space as possible. | |
| 1957 if (CMSCompactWhenClearAllSoftRefs) { | |
| 1958 // Default: The rationale is that in this case either | |
| 1959 // we are past the final marking phase, in which case | |
| 1960 // we'd have to start over, or so little has been done | |
| 1961 // that there's little point in saving that work. Compaction | |
| 1962 // appears to be the sensible choice in either case. | |
| 1963 *should_compact = true; | |
| 1964 } else { | |
| 1965 // We have been asked to clear all soft refs, but not to | |
| 1966 // compact. Make sure that we aren't past the final checkpoint | |
| 1967 // phase, for that is where we process soft refs. If we are already | |
| 1968 // past that phase, we'll need to redo the refs discovery phase and | |
| 1969 // if necessary clear soft refs that weren't previously | |
| 1970 // cleared. We do so by remembering the phase in which | |
| 1971 // we came in, and if we are past the refs processing | |
| 1972 // phase, we'll choose to just redo the mark-sweep | |
| 1973 // collection from scratch. | |
| 1974 if (_collectorState > FinalMarking) { | |
| 1975 // We are past the refs processing phase; | |
| 1976 // start over and do a fresh synchronous CMS cycle | |
| 1977 _collectorState = Resetting; // skip to reset to start new cycle | |
| 1978 reset(false /* == !asynch */); | |
| 1979 *should_start_over = true; | |
| 1980 } // else we can continue a possibly ongoing current cycle | |
| 1981 } | |
| 1982 } | |
| 1983 } | |
| 1984 | |
| 1985 // A work method used by the foreground collector to do | |
| 1986 // a mark-sweep-compact. | |
| 1987 void CMSCollector::do_compaction_work(bool clear_all_soft_refs) { | |
| 1988 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 1989 TraceTime t("CMS:MSC ", PrintGCDetails && Verbose, true, gclog_or_tty); | |
| 1990 if (PrintGC && Verbose && !(GCCause::is_user_requested_gc(gch->gc_cause()))) { | |
| 1991 gclog_or_tty->print_cr("Compact ConcurrentMarkSweepGeneration after %d " | |
| 1992 "collections passed to foreground collector", _full_gcs_since_conc_gc); | |
| 1993 } | |
| 1994 | |
| 1995 // Sample collection interval time and reset for collection pause. | |
| 1996 if (UseAdaptiveSizePolicy) { | |
| 1997 size_policy()->msc_collection_begin(); | |
| 1998 } | |
| 1999 | |
| 2000 // Temporarily widen the span of the weak reference processing to | |
| 2001 // the entire heap. | |
| 2002 MemRegion new_span(GenCollectedHeap::heap()->reserved_region()); | |
| 2003 ReferenceProcessorSpanMutator x(ref_processor(), new_span); | |
| 2004 | |
| 2005 // Temporarily, clear the "is_alive_non_header" field of the | |
| 2006 // reference processor. | |
| 2007 ReferenceProcessorIsAliveMutator y(ref_processor(), NULL); | |
| 2008 | |
| 2009 // Temporarily make reference _processing_ single threaded (non-MT). | |
| 2010 ReferenceProcessorMTProcMutator z(ref_processor(), false); | |
| 2011 | |
| 2012 // Temporarily make refs discovery atomic | |
| 2013 ReferenceProcessorAtomicMutator w(ref_processor(), true); | |
| 2014 | |
| 2015 ref_processor()->set_enqueuing_is_done(false); | |
| 2016 ref_processor()->enable_discovery(); | |
|
457
27a80744a83b
6778647: snap(), snap_policy() should be renamed setup(), setup_policy()
ysr
parents:
453
diff
changeset
|
2017 ref_processor()->setup_policy(clear_all_soft_refs); |
| 0 | 2018 // If an asynchronous collection finishes, the _modUnionTable is |
| 2019 // all clear. If we are assuming the collection from an asynchronous | |
| 2020 // collection, clear the _modUnionTable. | |
| 2021 assert(_collectorState != Idling || _modUnionTable.isAllClear(), | |
| 2022 "_modUnionTable should be clear if the baton was not passed"); | |
| 2023 _modUnionTable.clear_all(); | |
| 2024 | |
| 2025 // We must adjust the allocation statistics being maintained | |
| 2026 // in the free list space. We do so by reading and clearing | |
| 2027 // the sweep timer and updating the block flux rate estimates below. | |
|
1145
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
2028 assert(!_intra_sweep_timer.is_active(), "_intra_sweep_timer should be inactive"); |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
2029 if (_inter_sweep_timer.is_active()) { |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
2030 _inter_sweep_timer.stop(); |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
2031 // Note that we do not use this sample to update the _inter_sweep_estimate. |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
2032 _cmsGen->cmsSpace()->beginSweepFLCensus((float)(_inter_sweep_timer.seconds()), |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
2033 _inter_sweep_estimate.padded_average(), |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
2034 _intra_sweep_estimate.padded_average()); |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
2035 } |
| 0 | 2036 |
|
1703
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
2037 { |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
2038 TraceCMSMemoryManagerStats(); |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
2039 } |
| 0 | 2040 GenMarkSweep::invoke_at_safepoint(_cmsGen->level(), |
| 2041 ref_processor(), clear_all_soft_refs); | |
| 2042 #ifdef ASSERT | |
| 2043 CompactibleFreeListSpace* cms_space = _cmsGen->cmsSpace(); | |
| 2044 size_t free_size = cms_space->free(); | |
| 2045 assert(free_size == | |
| 2046 pointer_delta(cms_space->end(), cms_space->compaction_top()) | |
| 2047 * HeapWordSize, | |
| 2048 "All the free space should be compacted into one chunk at top"); | |
| 2049 assert(cms_space->dictionary()->totalChunkSize( | |
| 2050 debug_only(cms_space->freelistLock())) == 0 || | |
| 2051 cms_space->totalSizeInIndexedFreeLists() == 0, | |
| 2052 "All the free space should be in a single chunk"); | |
| 2053 size_t num = cms_space->totalCount(); | |
| 2054 assert((free_size == 0 && num == 0) || | |
| 2055 (free_size > 0 && (num == 1 || num == 2)), | |
| 2056 "There should be at most 2 free chunks after compaction"); | |
| 2057 #endif // ASSERT | |
| 2058 _collectorState = Resetting; | |
| 2059 assert(_restart_addr == NULL, | |
| 2060 "Should have been NULL'd before baton was passed"); | |
| 2061 reset(false /* == !asynch */); | |
| 2062 _cmsGen->reset_after_compaction(); | |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
2063 _concurrent_cycles_since_last_unload = 0; |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
2064 |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
2065 if (verifying() && !should_unload_classes()) { |
| 0 | 2066 perm_gen_verify_bit_map()->clear_all(); |
| 2067 } | |
| 2068 | |
| 2069 // Clear any data recorded in the PLAB chunk arrays. | |
| 2070 if (_survivor_plab_array != NULL) { | |
| 2071 reset_survivor_plab_arrays(); | |
| 2072 } | |
| 2073 | |
| 2074 // Adjust the per-size allocation stats for the next epoch. | |
|
1145
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
2075 _cmsGen->cmsSpace()->endSweepFLCensus(sweep_count() /* fake */); |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
2076 // Restart the "inter sweep timer" for the next epoch. |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
2077 _inter_sweep_timer.reset(); |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
2078 _inter_sweep_timer.start(); |
| 0 | 2079 |
| 2080 // Sample collection pause time and reset for collection interval. | |
| 2081 if (UseAdaptiveSizePolicy) { | |
| 2082 size_policy()->msc_collection_end(gch->gc_cause()); | |
| 2083 } | |
| 2084 | |
| 2085 // For a mark-sweep-compact, compute_new_size() will be called | |
| 2086 // in the heap's do_collection() method. | |
| 2087 } | |
| 2088 | |
| 2089 // A work method used by the foreground collector to do | |
| 2090 // a mark-sweep, after taking over from a possibly on-going | |
| 2091 // concurrent mark-sweep collection. | |
| 2092 void CMSCollector::do_mark_sweep_work(bool clear_all_soft_refs, | |
| 2093 CollectorState first_state, bool should_start_over) { | |
| 2094 if (PrintGC && Verbose) { | |
| 2095 gclog_or_tty->print_cr("Pass concurrent collection to foreground " | |
| 2096 "collector with count %d", | |
| 2097 _full_gcs_since_conc_gc); | |
| 2098 } | |
| 2099 switch (_collectorState) { | |
| 2100 case Idling: | |
| 2101 if (first_state == Idling || should_start_over) { | |
| 2102 // The background GC was not active, or should | |
| 2103 // restarted from scratch; start the cycle. | |
| 2104 _collectorState = InitialMarking; | |
| 2105 } | |
| 2106 // If first_state was not Idling, then a background GC | |
| 2107 // was in progress and has now finished. No need to do it | |
| 2108 // again. Leave the state as Idling. | |
| 2109 break; | |
| 2110 case Precleaning: | |
| 2111 // In the foreground case don't do the precleaning since | |
| 2112 // it is not done concurrently and there is extra work | |
| 2113 // required. | |
| 2114 _collectorState = FinalMarking; | |
| 2115 } | |
| 2116 if (PrintGCDetails && | |
| 2117 (_collectorState > Idling || | |
| 2118 !GCCause::is_user_requested_gc(GenCollectedHeap::heap()->gc_cause()))) { | |
| 2119 gclog_or_tty->print(" (concurrent mode failure)"); | |
| 2120 } | |
| 2121 collect_in_foreground(clear_all_soft_refs); | |
| 2122 | |
| 2123 // For a mark-sweep, compute_new_size() will be called | |
| 2124 // in the heap's do_collection() method. | |
| 2125 } | |
| 2126 | |
| 2127 | |
| 2128 void CMSCollector::getFreelistLocks() const { | |
| 2129 // Get locks for all free lists in all generations that this | |
| 2130 // collector is responsible for | |
| 2131 _cmsGen->freelistLock()->lock_without_safepoint_check(); | |
| 2132 _permGen->freelistLock()->lock_without_safepoint_check(); | |
| 2133 } | |
| 2134 | |
| 2135 void CMSCollector::releaseFreelistLocks() const { | |
| 2136 // Release locks for all free lists in all generations that this | |
| 2137 // collector is responsible for | |
| 2138 _cmsGen->freelistLock()->unlock(); | |
| 2139 _permGen->freelistLock()->unlock(); | |
| 2140 } | |
| 2141 | |
| 2142 bool CMSCollector::haveFreelistLocks() const { | |
| 2143 // Check locks for all free lists in all generations that this | |
| 2144 // collector is responsible for | |
| 2145 assert_lock_strong(_cmsGen->freelistLock()); | |
| 2146 assert_lock_strong(_permGen->freelistLock()); | |
| 2147 PRODUCT_ONLY(ShouldNotReachHere()); | |
| 2148 return true; | |
| 2149 } | |
| 2150 | |
| 2151 // A utility class that is used by the CMS collector to | |
| 2152 // temporarily "release" the foreground collector from its | |
| 2153 // usual obligation to wait for the background collector to | |
| 2154 // complete an ongoing phase before proceeding. | |
| 2155 class ReleaseForegroundGC: public StackObj { | |
| 2156 private: | |
| 2157 CMSCollector* _c; | |
| 2158 public: | |
| 2159 ReleaseForegroundGC(CMSCollector* c) : _c(c) { | |
| 2160 assert(_c->_foregroundGCShouldWait, "Else should not need to call"); | |
| 2161 MutexLockerEx x(CGC_lock, Mutex::_no_safepoint_check_flag); | |
| 2162 // allow a potentially blocked foreground collector to proceed | |
| 2163 _c->_foregroundGCShouldWait = false; | |
| 2164 if (_c->_foregroundGCIsActive) { | |
| 2165 CGC_lock->notify(); | |
| 2166 } | |
| 2167 assert(!ConcurrentMarkSweepThread::cms_thread_has_cms_token(), | |
| 2168 "Possible deadlock"); | |
| 2169 } | |
| 2170 | |
| 2171 ~ReleaseForegroundGC() { | |
| 2172 assert(!_c->_foregroundGCShouldWait, "Usage protocol violation?"); | |
| 2173 MutexLockerEx x(CGC_lock, Mutex::_no_safepoint_check_flag); | |
| 2174 _c->_foregroundGCShouldWait = true; | |
| 2175 } | |
| 2176 }; | |
| 2177 | |
| 2178 // There are separate collect_in_background and collect_in_foreground because of | |
| 2179 // the different locking requirements of the background collector and the | |
| 2180 // foreground collector. There was originally an attempt to share | |
| 2181 // one "collect" method between the background collector and the foreground | |
| 2182 // collector but the if-then-else required made it cleaner to have | |
| 2183 // separate methods. | |
| 2184 void CMSCollector::collect_in_background(bool clear_all_soft_refs) { | |
| 2185 assert(Thread::current()->is_ConcurrentGC_thread(), | |
| 2186 "A CMS asynchronous collection is only allowed on a CMS thread."); | |
| 2187 | |
| 2188 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 2189 { | |
| 2190 bool safepoint_check = Mutex::_no_safepoint_check_flag; | |
| 2191 MutexLockerEx hl(Heap_lock, safepoint_check); | |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
2192 FreelistLocker fll(this); |
| 0 | 2193 MutexLockerEx x(CGC_lock, safepoint_check); |
| 2194 if (_foregroundGCIsActive || !UseAsyncConcMarkSweepGC) { | |
| 2195 // The foreground collector is active or we're | |
| 2196 // not using asynchronous collections. Skip this | |
| 2197 // background collection. | |
| 2198 assert(!_foregroundGCShouldWait, "Should be clear"); | |
| 2199 return; | |
| 2200 } else { | |
| 2201 assert(_collectorState == Idling, "Should be idling before start."); | |
| 2202 _collectorState = InitialMarking; | |
| 2203 // Reset the expansion cause, now that we are about to begin | |
| 2204 // a new cycle. | |
| 2205 clear_expansion_cause(); | |
| 2206 } | |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
2207 // Decide if we want to enable class unloading as part of the |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
2208 // ensuing concurrent GC cycle. |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
2209 update_should_unload_classes(); |
| 0 | 2210 _full_gc_requested = false; // acks all outstanding full gc requests |
| 2211 // Signal that we are about to start a collection | |
| 2212 gch->increment_total_full_collections(); // ... starting a collection cycle | |
| 2213 _collection_count_start = gch->total_full_collections(); | |
| 2214 } | |
| 2215 | |
| 2216 // Used for PrintGC | |
| 2217 size_t prev_used; | |
| 2218 if (PrintGC && Verbose) { | |
| 2219 prev_used = _cmsGen->used(); // XXXPERM | |
| 2220 } | |
| 2221 | |
| 2222 // The change of the collection state is normally done at this level; | |
| 2223 // the exceptions are phases that are executed while the world is | |
| 2224 // stopped. For those phases the change of state is done while the | |
| 2225 // world is stopped. For baton passing purposes this allows the | |
| 2226 // background collector to finish the phase and change state atomically. | |
| 2227 // The foreground collector cannot wait on a phase that is done | |
| 2228 // while the world is stopped because the foreground collector already | |
| 2229 // has the world stopped and would deadlock. | |
| 2230 while (_collectorState != Idling) { | |
| 2231 if (TraceCMSState) { | |
| 2232 gclog_or_tty->print_cr("Thread " INTPTR_FORMAT " in CMS state %d", | |
| 2233 Thread::current(), _collectorState); | |
| 2234 } | |
| 2235 // The foreground collector | |
| 2236 // holds the Heap_lock throughout its collection. | |
| 2237 // holds the CMS token (but not the lock) | |
| 2238 // except while it is waiting for the background collector to yield. | |
| 2239 // | |
| 2240 // The foreground collector should be blocked (not for long) | |
| 2241 // if the background collector is about to start a phase | |
| 2242 // executed with world stopped. If the background | |
| 2243 // collector has already started such a phase, the | |
| 2244 // foreground collector is blocked waiting for the | |
| 2245 // Heap_lock. The stop-world phases (InitialMarking and FinalMarking) | |
| 2246 // are executed in the VM thread. | |
| 2247 // | |
| 2248 // The locking order is | |
| 2249 // PendingListLock (PLL) -- if applicable (FinalMarking) | |
| 2250 // Heap_lock (both this & PLL locked in VM_CMS_Operation::prologue()) | |
| 2251 // CMS token (claimed in | |
| 2252 // stop_world_and_do() --> | |
| 2253 // safepoint_synchronize() --> | |
| 2254 // CMSThread::synchronize()) | |
| 2255 | |
| 2256 { | |
| 2257 // Check if the FG collector wants us to yield. | |
| 2258 CMSTokenSync x(true); // is cms thread | |
| 2259 if (waitForForegroundGC()) { | |
| 2260 // We yielded to a foreground GC, nothing more to be | |
| 2261 // done this round. | |
| 2262 assert(_foregroundGCShouldWait == false, "We set it to false in " | |
| 2263 "waitForForegroundGC()"); | |
| 2264 if (TraceCMSState) { | |
| 2265 gclog_or_tty->print_cr("CMS Thread " INTPTR_FORMAT | |
| 2266 " exiting collection CMS state %d", | |
| 2267 Thread::current(), _collectorState); | |
| 2268 } | |
| 2269 return; | |
| 2270 } else { | |
| 2271 // The background collector can run but check to see if the | |
| 2272 // foreground collector has done a collection while the | |
| 2273 // background collector was waiting to get the CGC_lock | |
| 2274 // above. If yes, break so that _foregroundGCShouldWait | |
| 2275 // is cleared before returning. | |
| 2276 if (_collectorState == Idling) { | |
| 2277 break; | |
| 2278 } | |
| 2279 } | |
| 2280 } | |
| 2281 | |
| 2282 assert(_foregroundGCShouldWait, "Foreground collector, if active, " | |
| 2283 "should be waiting"); | |
| 2284 | |
| 2285 switch (_collectorState) { | |
| 2286 case InitialMarking: | |
| 2287 { | |
| 2288 ReleaseForegroundGC x(this); | |
| 2289 stats().record_cms_begin(); | |
| 2290 | |
| 2291 VM_CMS_Initial_Mark initial_mark_op(this); | |
| 2292 VMThread::execute(&initial_mark_op); | |
| 2293 } | |
| 2294 // The collector state may be any legal state at this point | |
| 2295 // since the background collector may have yielded to the | |
| 2296 // foreground collector. | |
| 2297 break; | |
| 2298 case Marking: | |
| 2299 // initial marking in checkpointRootsInitialWork has been completed | |
| 2300 if (markFromRoots(true)) { // we were successful | |
| 2301 assert(_collectorState == Precleaning, "Collector state should " | |
| 2302 "have changed"); | |
| 2303 } else { | |
| 2304 assert(_foregroundGCIsActive, "Internal state inconsistency"); | |
| 2305 } | |
| 2306 break; | |
| 2307 case Precleaning: | |
| 2308 if (UseAdaptiveSizePolicy) { | |
| 2309 size_policy()->concurrent_precleaning_begin(); | |
| 2310 } | |
| 2311 // marking from roots in markFromRoots has been completed | |
| 2312 preclean(); | |
| 2313 if (UseAdaptiveSizePolicy) { | |
| 2314 size_policy()->concurrent_precleaning_end(); | |
| 2315 } | |
| 2316 assert(_collectorState == AbortablePreclean || | |
| 2317 _collectorState == FinalMarking, | |
| 2318 "Collector state should have changed"); | |
| 2319 break; | |
| 2320 case AbortablePreclean: | |
| 2321 if (UseAdaptiveSizePolicy) { | |
| 2322 size_policy()->concurrent_phases_resume(); | |
| 2323 } | |
| 2324 abortable_preclean(); | |
| 2325 if (UseAdaptiveSizePolicy) { | |
| 2326 size_policy()->concurrent_precleaning_end(); | |
| 2327 } | |
| 2328 assert(_collectorState == FinalMarking, "Collector state should " | |
| 2329 "have changed"); | |
| 2330 break; | |
| 2331 case FinalMarking: | |
| 2332 { | |
| 2333 ReleaseForegroundGC x(this); | |
| 2334 | |
| 2335 VM_CMS_Final_Remark final_remark_op(this); | |
| 2336 VMThread::execute(&final_remark_op); | |
| 935 | 2337 } |
| 0 | 2338 assert(_foregroundGCShouldWait, "block post-condition"); |
| 2339 break; | |
| 2340 case Sweeping: | |
| 2341 if (UseAdaptiveSizePolicy) { | |
| 2342 size_policy()->concurrent_sweeping_begin(); | |
| 2343 } | |
| 2344 // final marking in checkpointRootsFinal has been completed | |
| 2345 sweep(true); | |
| 2346 assert(_collectorState == Resizing, "Collector state change " | |
| 2347 "to Resizing must be done under the free_list_lock"); | |
| 2348 _full_gcs_since_conc_gc = 0; | |
| 2349 | |
| 2350 // Stop the timers for adaptive size policy for the concurrent phases | |
| 2351 if (UseAdaptiveSizePolicy) { | |
| 2352 size_policy()->concurrent_sweeping_end(); | |
| 2353 size_policy()->concurrent_phases_end(gch->gc_cause(), | |
| 2354 gch->prev_gen(_cmsGen)->capacity(), | |
| 2355 _cmsGen->free()); | |
| 2356 } | |
| 2357 | |
| 2358 case Resizing: { | |
| 2359 // Sweeping has been completed... | |
| 2360 // At this point the background collection has completed. | |
| 2361 // Don't move the call to compute_new_size() down | |
| 2362 // into code that might be executed if the background | |
| 2363 // collection was preempted. | |
| 2364 { | |
| 2365 ReleaseForegroundGC x(this); // unblock FG collection | |
| 2366 MutexLockerEx y(Heap_lock, Mutex::_no_safepoint_check_flag); | |
| 2367 CMSTokenSync z(true); // not strictly needed. | |
| 2368 if (_collectorState == Resizing) { | |
| 2369 compute_new_size(); | |
| 2370 _collectorState = Resetting; | |
| 2371 } else { | |
| 2372 assert(_collectorState == Idling, "The state should only change" | |
| 2373 " because the foreground collector has finished the collection"); | |
| 2374 } | |
| 2375 } | |
| 2376 break; | |
| 2377 } | |
| 2378 case Resetting: | |
| 2379 // CMS heap resizing has been completed | |
| 2380 reset(true); | |
| 2381 assert(_collectorState == Idling, "Collector state should " | |
| 2382 "have changed"); | |
| 2383 stats().record_cms_end(); | |
| 2384 // Don't move the concurrent_phases_end() and compute_new_size() | |
| 2385 // calls to here because a preempted background collection | |
| 2386 // has it's state set to "Resetting". | |
| 2387 break; | |
| 2388 case Idling: | |
| 2389 default: | |
| 2390 ShouldNotReachHere(); | |
| 2391 break; | |
| 2392 } | |
| 2393 if (TraceCMSState) { | |
| 2394 gclog_or_tty->print_cr(" Thread " INTPTR_FORMAT " done - next CMS state %d", | |
| 2395 Thread::current(), _collectorState); | |
| 2396 } | |
| 2397 assert(_foregroundGCShouldWait, "block post-condition"); | |
| 2398 } | |
| 2399 | |
| 2400 // Should this be in gc_epilogue? | |
| 2401 collector_policy()->counters()->update_counters(); | |
| 2402 | |
| 2403 { | |
| 2404 // Clear _foregroundGCShouldWait and, in the event that the | |
| 2405 // foreground collector is waiting, notify it, before | |
| 2406 // returning. | |
| 2407 MutexLockerEx x(CGC_lock, Mutex::_no_safepoint_check_flag); | |
| 2408 _foregroundGCShouldWait = false; | |
| 2409 if (_foregroundGCIsActive) { | |
| 2410 CGC_lock->notify(); | |
| 2411 } | |
| 2412 assert(!ConcurrentMarkSweepThread::cms_thread_has_cms_token(), | |
| 2413 "Possible deadlock"); | |
| 2414 } | |
| 2415 if (TraceCMSState) { | |
| 2416 gclog_or_tty->print_cr("CMS Thread " INTPTR_FORMAT | |
| 2417 " exiting collection CMS state %d", | |
| 2418 Thread::current(), _collectorState); | |
| 2419 } | |
| 2420 if (PrintGC && Verbose) { | |
| 2421 _cmsGen->print_heap_change(prev_used); | |
| 2422 } | |
| 2423 } | |
| 2424 | |
| 2425 void CMSCollector::collect_in_foreground(bool clear_all_soft_refs) { | |
| 2426 assert(_foregroundGCIsActive && !_foregroundGCShouldWait, | |
| 2427 "Foreground collector should be waiting, not executing"); | |
| 2428 assert(Thread::current()->is_VM_thread(), "A foreground collection" | |
| 2429 "may only be done by the VM Thread with the world stopped"); | |
| 2430 assert(ConcurrentMarkSweepThread::vm_thread_has_cms_token(), | |
| 2431 "VM thread should have CMS token"); | |
| 2432 | |
| 2433 NOT_PRODUCT(TraceTime t("CMS:MS (foreground) ", PrintGCDetails && Verbose, | |
| 2434 true, gclog_or_tty);) | |
| 2435 if (UseAdaptiveSizePolicy) { | |
| 2436 size_policy()->ms_collection_begin(); | |
| 2437 } | |
| 2438 COMPILER2_PRESENT(DerivedPointerTableDeactivate dpt_deact); | |
| 2439 | |
| 2440 HandleMark hm; // Discard invalid handles created during verification | |
| 2441 | |
| 2442 if (VerifyBeforeGC && | |
| 2443 GenCollectedHeap::heap()->total_collections() >= VerifyGCStartAt) { | |
| 2444 Universe::verify(true); | |
| 2445 } | |
| 2446 | |
|
453
c96030fff130
6684579: SoftReference processing can be made more efficient
ysr
parents:
452
diff
changeset
|
2447 // Snapshot the soft reference policy to be used in this collection cycle. |
|
457
27a80744a83b
6778647: snap(), snap_policy() should be renamed setup(), setup_policy()
ysr
parents:
453
diff
changeset
|
2448 ref_processor()->setup_policy(clear_all_soft_refs); |
|
453
c96030fff130
6684579: SoftReference processing can be made more efficient
ysr
parents:
452
diff
changeset
|
2449 |
| 0 | 2450 bool init_mark_was_synchronous = false; // until proven otherwise |
| 2451 while (_collectorState != Idling) { | |
| 2452 if (TraceCMSState) { | |
| 2453 gclog_or_tty->print_cr("Thread " INTPTR_FORMAT " in CMS state %d", | |
| 2454 Thread::current(), _collectorState); | |
| 2455 } | |
| 2456 switch (_collectorState) { | |
| 2457 case InitialMarking: | |
| 2458 init_mark_was_synchronous = true; // fact to be exploited in re-mark | |
| 2459 checkpointRootsInitial(false); | |
| 2460 assert(_collectorState == Marking, "Collector state should have changed" | |
| 2461 " within checkpointRootsInitial()"); | |
| 2462 break; | |
| 2463 case Marking: | |
| 2464 // initial marking in checkpointRootsInitialWork has been completed | |
| 2465 if (VerifyDuringGC && | |
| 2466 GenCollectedHeap::heap()->total_collections() >= VerifyGCStartAt) { | |
| 2467 gclog_or_tty->print("Verify before initial mark: "); | |
| 2468 Universe::verify(true); | |
| 2469 } | |
| 2470 { | |
| 2471 bool res = markFromRoots(false); | |
| 2472 assert(res && _collectorState == FinalMarking, "Collector state should " | |
| 2473 "have changed"); | |
| 2474 break; | |
| 2475 } | |
| 2476 case FinalMarking: | |
| 2477 if (VerifyDuringGC && | |
| 2478 GenCollectedHeap::heap()->total_collections() >= VerifyGCStartAt) { | |
| 2479 gclog_or_tty->print("Verify before re-mark: "); | |
| 2480 Universe::verify(true); | |
| 2481 } | |
| 2482 checkpointRootsFinal(false, clear_all_soft_refs, | |
| 2483 init_mark_was_synchronous); | |
| 2484 assert(_collectorState == Sweeping, "Collector state should not " | |
| 2485 "have changed within checkpointRootsFinal()"); | |
| 2486 break; | |
| 2487 case Sweeping: | |
| 2488 // final marking in checkpointRootsFinal has been completed | |
| 2489 if (VerifyDuringGC && | |
| 2490 GenCollectedHeap::heap()->total_collections() >= VerifyGCStartAt) { | |
| 2491 gclog_or_tty->print("Verify before sweep: "); | |
| 2492 Universe::verify(true); | |
| 2493 } | |
| 2494 sweep(false); | |
| 2495 assert(_collectorState == Resizing, "Incorrect state"); | |
| 2496 break; | |
| 2497 case Resizing: { | |
| 2498 // Sweeping has been completed; the actual resize in this case | |
| 2499 // is done separately; nothing to be done in this state. | |
| 2500 _collectorState = Resetting; | |
| 2501 break; | |
| 2502 } | |
| 2503 case Resetting: | |
| 2504 // The heap has been resized. | |
| 2505 if (VerifyDuringGC && | |
| 2506 GenCollectedHeap::heap()->total_collections() >= VerifyGCStartAt) { | |
| 2507 gclog_or_tty->print("Verify before reset: "); | |
| 2508 Universe::verify(true); | |
| 2509 } | |
| 2510 reset(false); | |
| 2511 assert(_collectorState == Idling, "Collector state should " | |
| 2512 "have changed"); | |
| 2513 break; | |
| 2514 case Precleaning: | |
| 2515 case AbortablePreclean: | |
| 2516 // Elide the preclean phase | |
| 2517 _collectorState = FinalMarking; | |
| 2518 break; | |
| 2519 default: | |
| 2520 ShouldNotReachHere(); | |
| 2521 } | |
| 2522 if (TraceCMSState) { | |
| 2523 gclog_or_tty->print_cr(" Thread " INTPTR_FORMAT " done - next CMS state %d", | |
| 2524 Thread::current(), _collectorState); | |
| 2525 } | |
| 2526 } | |
| 2527 | |
| 2528 if (UseAdaptiveSizePolicy) { | |
| 2529 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 2530 size_policy()->ms_collection_end(gch->gc_cause()); | |
| 2531 } | |
| 2532 | |
| 2533 if (VerifyAfterGC && | |
| 2534 GenCollectedHeap::heap()->total_collections() >= VerifyGCStartAt) { | |
| 2535 Universe::verify(true); | |
| 2536 } | |
| 2537 if (TraceCMSState) { | |
| 2538 gclog_or_tty->print_cr("CMS Thread " INTPTR_FORMAT | |
| 2539 " exiting collection CMS state %d", | |
| 2540 Thread::current(), _collectorState); | |
| 2541 } | |
| 2542 } | |
| 2543 | |
| 2544 bool CMSCollector::waitForForegroundGC() { | |
| 2545 bool res = false; | |
| 2546 assert(ConcurrentMarkSweepThread::cms_thread_has_cms_token(), | |
| 2547 "CMS thread should have CMS token"); | |
| 2548 // Block the foreground collector until the | |
| 2549 // background collectors decides whether to | |
| 2550 // yield. | |
| 2551 MutexLockerEx x(CGC_lock, Mutex::_no_safepoint_check_flag); | |
| 2552 _foregroundGCShouldWait = true; | |
| 2553 if (_foregroundGCIsActive) { | |
| 2554 // The background collector yields to the | |
| 2555 // foreground collector and returns a value | |
| 2556 // indicating that it has yielded. The foreground | |
| 2557 // collector can proceed. | |
| 2558 res = true; | |
| 2559 _foregroundGCShouldWait = false; | |
| 2560 ConcurrentMarkSweepThread::clear_CMS_flag( | |
| 2561 ConcurrentMarkSweepThread::CMS_cms_has_token); | |
| 2562 ConcurrentMarkSweepThread::set_CMS_flag( | |
| 2563 ConcurrentMarkSweepThread::CMS_cms_wants_token); | |
| 2564 // Get a possibly blocked foreground thread going | |
| 2565 CGC_lock->notify(); | |
| 2566 if (TraceCMSState) { | |
| 2567 gclog_or_tty->print_cr("CMS Thread " INTPTR_FORMAT " waiting at CMS state %d", | |
| 2568 Thread::current(), _collectorState); | |
| 2569 } | |
| 2570 while (_foregroundGCIsActive) { | |
| 2571 CGC_lock->wait(Mutex::_no_safepoint_check_flag); | |
| 2572 } | |
| 2573 ConcurrentMarkSweepThread::set_CMS_flag( | |
| 2574 ConcurrentMarkSweepThread::CMS_cms_has_token); | |
| 2575 ConcurrentMarkSweepThread::clear_CMS_flag( | |
| 2576 ConcurrentMarkSweepThread::CMS_cms_wants_token); | |
| 2577 } | |
| 2578 if (TraceCMSState) { | |
| 2579 gclog_or_tty->print_cr("CMS Thread " INTPTR_FORMAT " continuing at CMS state %d", | |
| 2580 Thread::current(), _collectorState); | |
| 2581 } | |
| 2582 return res; | |
| 2583 } | |
| 2584 | |
| 2585 // Because of the need to lock the free lists and other structures in | |
| 2586 // the collector, common to all the generations that the collector is | |
| 2587 // collecting, we need the gc_prologues of individual CMS generations | |
| 2588 // delegate to their collector. It may have been simpler had the | |
| 2589 // current infrastructure allowed one to call a prologue on a | |
| 2590 // collector. In the absence of that we have the generation's | |
| 2591 // prologue delegate to the collector, which delegates back | |
| 2592 // some "local" work to a worker method in the individual generations | |
| 2593 // that it's responsible for collecting, while itself doing any | |
| 2594 // work common to all generations it's responsible for. A similar | |
| 2595 // comment applies to the gc_epilogue()'s. | |
| 2596 // The role of the varaible _between_prologue_and_epilogue is to | |
| 2597 // enforce the invocation protocol. | |
| 2598 void CMSCollector::gc_prologue(bool full) { | |
| 2599 // Call gc_prologue_work() for each CMSGen and PermGen that | |
| 2600 // we are responsible for. | |
| 2601 | |
| 2602 // The following locking discipline assumes that we are only called | |
| 2603 // when the world is stopped. | |
| 2604 assert(SafepointSynchronize::is_at_safepoint(), "world is stopped assumption"); | |
| 2605 | |
| 2606 // The CMSCollector prologue must call the gc_prologues for the | |
| 2607 // "generations" (including PermGen if any) that it's responsible | |
| 2608 // for. | |
| 2609 | |
| 2610 assert( Thread::current()->is_VM_thread() | |
| 2611 || ( CMSScavengeBeforeRemark | |
| 2612 && Thread::current()->is_ConcurrentGC_thread()), | |
| 2613 "Incorrect thread type for prologue execution"); | |
| 2614 | |
| 2615 if (_between_prologue_and_epilogue) { | |
| 2616 // We have already been invoked; this is a gc_prologue delegation | |
| 2617 // from yet another CMS generation that we are responsible for, just | |
| 2618 // ignore it since all relevant work has already been done. | |
| 2619 return; | |
| 2620 } | |
| 2621 | |
| 2622 // set a bit saying prologue has been called; cleared in epilogue | |
| 2623 _between_prologue_and_epilogue = true; | |
| 2624 // Claim locks for common data structures, then call gc_prologue_work() | |
| 2625 // for each CMSGen and PermGen that we are responsible for. | |
| 2626 | |
| 2627 getFreelistLocks(); // gets free list locks on constituent spaces | |
| 2628 bitMapLock()->lock_without_safepoint_check(); | |
| 2629 | |
| 2630 // Should call gc_prologue_work() for all cms gens we are responsible for | |
| 2631 bool registerClosure = _collectorState >= Marking | |
| 2632 && _collectorState < Sweeping; | |
| 2633 ModUnionClosure* muc = ParallelGCThreads > 0 ? &_modUnionClosurePar | |
| 2634 : &_modUnionClosure; | |
| 2635 _cmsGen->gc_prologue_work(full, registerClosure, muc); | |
| 2636 _permGen->gc_prologue_work(full, registerClosure, muc); | |
| 2637 | |
| 2638 if (!full) { | |
| 2639 stats().record_gc0_begin(); | |
| 2640 } | |
| 2641 } | |
| 2642 | |
| 2643 void ConcurrentMarkSweepGeneration::gc_prologue(bool full) { | |
| 2644 // Delegate to CMScollector which knows how to coordinate between | |
| 2645 // this and any other CMS generations that it is responsible for | |
| 2646 // collecting. | |
| 2647 collector()->gc_prologue(full); | |
| 2648 } | |
| 2649 | |
| 2650 // This is a "private" interface for use by this generation's CMSCollector. | |
| 2651 // Not to be called directly by any other entity (for instance, | |
| 2652 // GenCollectedHeap, which calls the "public" gc_prologue method above). | |
| 2653 void ConcurrentMarkSweepGeneration::gc_prologue_work(bool full, | |
| 2654 bool registerClosure, ModUnionClosure* modUnionClosure) { | |
| 2655 assert(!incremental_collection_failed(), "Shouldn't be set yet"); | |
| 2656 assert(cmsSpace()->preconsumptionDirtyCardClosure() == NULL, | |
| 2657 "Should be NULL"); | |
| 2658 if (registerClosure) { | |
| 2659 cmsSpace()->setPreconsumptionDirtyCardClosure(modUnionClosure); | |
| 2660 } | |
| 2661 cmsSpace()->gc_prologue(); | |
| 2662 // Clear stat counters | |
| 2663 NOT_PRODUCT( | |
| 2664 assert(_numObjectsPromoted == 0, "check"); | |
| 2665 assert(_numWordsPromoted == 0, "check"); | |
| 2666 if (Verbose && PrintGC) { | |
| 2667 gclog_or_tty->print("Allocated "SIZE_FORMAT" objects, " | |
| 2668 SIZE_FORMAT" bytes concurrently", | |
| 2669 _numObjectsAllocated, _numWordsAllocated*sizeof(HeapWord)); | |
| 2670 } | |
| 2671 _numObjectsAllocated = 0; | |
| 2672 _numWordsAllocated = 0; | |
| 2673 ) | |
| 2674 } | |
| 2675 | |
| 2676 void CMSCollector::gc_epilogue(bool full) { | |
| 2677 // The following locking discipline assumes that we are only called | |
| 2678 // when the world is stopped. | |
| 2679 assert(SafepointSynchronize::is_at_safepoint(), | |
| 2680 "world is stopped assumption"); | |
| 2681 | |
| 2682 // Currently the CMS epilogue (see CompactibleFreeListSpace) merely checks | |
| 2683 // if linear allocation blocks need to be appropriately marked to allow the | |
| 2684 // the blocks to be parsable. We also check here whether we need to nudge the | |
| 2685 // CMS collector thread to start a new cycle (if it's not already active). | |
| 2686 assert( Thread::current()->is_VM_thread() | |
| 2687 || ( CMSScavengeBeforeRemark | |
| 2688 && Thread::current()->is_ConcurrentGC_thread()), | |
| 2689 "Incorrect thread type for epilogue execution"); | |
| 2690 | |
| 2691 if (!_between_prologue_and_epilogue) { | |
| 2692 // We have already been invoked; this is a gc_epilogue delegation | |
| 2693 // from yet another CMS generation that we are responsible for, just | |
| 2694 // ignore it since all relevant work has already been done. | |
| 2695 return; | |
| 2696 } | |
| 2697 assert(haveFreelistLocks(), "must have freelist locks"); | |
| 2698 assert_lock_strong(bitMapLock()); | |
| 2699 | |
| 2700 _cmsGen->gc_epilogue_work(full); | |
| 2701 _permGen->gc_epilogue_work(full); | |
| 2702 | |
| 2703 if (_collectorState == AbortablePreclean || _collectorState == Precleaning) { | |
| 2704 // in case sampling was not already enabled, enable it | |
| 2705 _start_sampling = true; | |
| 2706 } | |
| 2707 // reset _eden_chunk_array so sampling starts afresh | |
| 2708 _eden_chunk_index = 0; | |
| 2709 | |
| 2710 size_t cms_used = _cmsGen->cmsSpace()->used(); | |
| 2711 size_t perm_used = _permGen->cmsSpace()->used(); | |
| 2712 | |
| 2713 // update performance counters - this uses a special version of | |
| 2714 // update_counters() that allows the utilization to be passed as a | |
| 2715 // parameter, avoiding multiple calls to used(). | |
| 2716 // | |
| 2717 _cmsGen->update_counters(cms_used); | |
| 2718 _permGen->update_counters(perm_used); | |
| 2719 | |
| 2720 if (CMSIncrementalMode) { | |
| 2721 icms_update_allocation_limits(); | |
| 2722 } | |
| 2723 | |
| 2724 bitMapLock()->unlock(); | |
| 2725 releaseFreelistLocks(); | |
| 2726 | |
| 2727 _between_prologue_and_epilogue = false; // ready for next cycle | |
| 2728 } | |
| 2729 | |
| 2730 void ConcurrentMarkSweepGeneration::gc_epilogue(bool full) { | |
| 2731 collector()->gc_epilogue(full); | |
| 2732 | |
| 2733 // Also reset promotion tracking in par gc thread states. | |
| 2734 if (ParallelGCThreads > 0) { | |
| 2735 for (uint i = 0; i < ParallelGCThreads; i++) { | |
|
1145
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
2736 _par_gc_thread_states[i]->promo.stopTrackingPromotions(i); |
| 0 | 2737 } |
| 2738 } | |
| 2739 } | |
| 2740 | |
| 2741 void ConcurrentMarkSweepGeneration::gc_epilogue_work(bool full) { | |
| 2742 assert(!incremental_collection_failed(), "Should have been cleared"); | |
| 2743 cmsSpace()->setPreconsumptionDirtyCardClosure(NULL); | |
| 2744 cmsSpace()->gc_epilogue(); | |
| 2745 // Print stat counters | |
| 2746 NOT_PRODUCT( | |
| 2747 assert(_numObjectsAllocated == 0, "check"); | |
| 2748 assert(_numWordsAllocated == 0, "check"); | |
| 2749 if (Verbose && PrintGC) { | |
| 2750 gclog_or_tty->print("Promoted "SIZE_FORMAT" objects, " | |
| 2751 SIZE_FORMAT" bytes", | |
| 2752 _numObjectsPromoted, _numWordsPromoted*sizeof(HeapWord)); | |
| 2753 } | |
| 2754 _numObjectsPromoted = 0; | |
| 2755 _numWordsPromoted = 0; | |
| 2756 ) | |
| 2757 | |
| 2758 if (PrintGC && Verbose) { | |
| 2759 // Call down the chain in contiguous_available needs the freelistLock | |
| 2760 // so print this out before releasing the freeListLock. | |
| 2761 gclog_or_tty->print(" Contiguous available "SIZE_FORMAT" bytes ", | |
| 2762 contiguous_available()); | |
| 2763 } | |
| 2764 } | |
| 2765 | |
| 2766 #ifndef PRODUCT | |
| 2767 bool CMSCollector::have_cms_token() { | |
| 2768 Thread* thr = Thread::current(); | |
| 2769 if (thr->is_VM_thread()) { | |
| 2770 return ConcurrentMarkSweepThread::vm_thread_has_cms_token(); | |
| 2771 } else if (thr->is_ConcurrentGC_thread()) { | |
| 2772 return ConcurrentMarkSweepThread::cms_thread_has_cms_token(); | |
| 2773 } else if (thr->is_GC_task_thread()) { | |
| 2774 return ConcurrentMarkSweepThread::vm_thread_has_cms_token() && | |
| 2775 ParGCRareEvent_lock->owned_by_self(); | |
| 2776 } | |
| 2777 return false; | |
| 2778 } | |
| 2779 #endif | |
| 2780 | |
| 2781 // Check reachability of the given heap address in CMS generation, | |
| 2782 // treating all other generations as roots. | |
| 2783 bool CMSCollector::is_cms_reachable(HeapWord* addr) { | |
| 2784 // We could "guarantee" below, rather than assert, but i'll | |
| 2785 // leave these as "asserts" so that an adventurous debugger | |
| 2786 // could try this in the product build provided some subset of | |
| 2787 // the conditions were met, provided they were intersted in the | |
| 2788 // results and knew that the computation below wouldn't interfere | |
| 2789 // with other concurrent computations mutating the structures | |
| 2790 // being read or written. | |
| 2791 assert(SafepointSynchronize::is_at_safepoint(), | |
| 2792 "Else mutations in object graph will make answer suspect"); | |
| 2793 assert(have_cms_token(), "Should hold cms token"); | |
| 2794 assert(haveFreelistLocks(), "must hold free list locks"); | |
| 2795 assert_lock_strong(bitMapLock()); | |
| 2796 | |
| 2797 // Clear the marking bit map array before starting, but, just | |
| 2798 // for kicks, first report if the given address is already marked | |
| 2799 gclog_or_tty->print_cr("Start: Address 0x%x is%s marked", addr, | |
| 2800 _markBitMap.isMarked(addr) ? "" : " not"); | |
| 2801 | |
| 2802 if (verify_after_remark()) { | |
| 2803 MutexLockerEx x(verification_mark_bm()->lock(), Mutex::_no_safepoint_check_flag); | |
| 2804 bool result = verification_mark_bm()->isMarked(addr); | |
| 2805 gclog_or_tty->print_cr("TransitiveMark: Address 0x%x %s marked", addr, | |
| 2806 result ? "IS" : "is NOT"); | |
| 2807 return result; | |
| 2808 } else { | |
| 2809 gclog_or_tty->print_cr("Could not compute result"); | |
| 2810 return false; | |
| 2811 } | |
| 2812 } | |
| 2813 | |
| 2814 //////////////////////////////////////////////////////// | |
| 2815 // CMS Verification Support | |
| 2816 //////////////////////////////////////////////////////// | |
| 2817 // Following the remark phase, the following invariant | |
| 2818 // should hold -- each object in the CMS heap which is | |
| 2819 // marked in markBitMap() should be marked in the verification_mark_bm(). | |
| 2820 | |
| 2821 class VerifyMarkedClosure: public BitMapClosure { | |
| 2822 CMSBitMap* _marks; | |
| 2823 bool _failed; | |
| 2824 | |
| 2825 public: | |
| 2826 VerifyMarkedClosure(CMSBitMap* bm): _marks(bm), _failed(false) {} | |
| 2827 | |
|
342
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
2828 bool do_bit(size_t offset) { |
| 0 | 2829 HeapWord* addr = _marks->offsetToHeapWord(offset); |
| 2830 if (!_marks->isMarked(addr)) { | |
|
1145
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
2831 oop(addr)->print_on(gclog_or_tty); |
| 0 | 2832 gclog_or_tty->print_cr(" ("INTPTR_FORMAT" should have been marked)", addr); |
| 2833 _failed = true; | |
| 2834 } | |
|
342
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
2835 return true; |
| 0 | 2836 } |
| 2837 | |
| 2838 bool failed() { return _failed; } | |
| 2839 }; | |
| 2840 | |
| 2841 bool CMSCollector::verify_after_remark() { | |
| 2842 gclog_or_tty->print(" [Verifying CMS Marking... "); | |
| 2843 MutexLockerEx ml(verification_mark_bm()->lock(), Mutex::_no_safepoint_check_flag); | |
| 2844 static bool init = false; | |
| 2845 | |
| 2846 assert(SafepointSynchronize::is_at_safepoint(), | |
| 2847 "Else mutations in object graph will make answer suspect"); | |
| 2848 assert(have_cms_token(), | |
| 2849 "Else there may be mutual interference in use of " | |
| 2850 " verification data structures"); | |
| 2851 assert(_collectorState > Marking && _collectorState <= Sweeping, | |
| 2852 "Else marking info checked here may be obsolete"); | |
| 2853 assert(haveFreelistLocks(), "must hold free list locks"); | |
| 2854 assert_lock_strong(bitMapLock()); | |
| 2855 | |
| 2856 | |
| 2857 // Allocate marking bit map if not already allocated | |
| 2858 if (!init) { // first time | |
| 2859 if (!verification_mark_bm()->allocate(_span)) { | |
| 2860 return false; | |
| 2861 } | |
| 2862 init = true; | |
| 2863 } | |
| 2864 | |
| 2865 assert(verification_mark_stack()->isEmpty(), "Should be empty"); | |
| 2866 | |
| 2867 // Turn off refs discovery -- so we will be tracing through refs. | |
| 2868 // This is as intended, because by this time | |
| 2869 // GC must already have cleared any refs that need to be cleared, | |
| 2870 // and traced those that need to be marked; moreover, | |
| 2871 // the marking done here is not going to intefere in any | |
| 2872 // way with the marking information used by GC. | |
| 2873 NoRefDiscovery no_discovery(ref_processor()); | |
| 2874 | |
| 2875 COMPILER2_PRESENT(DerivedPointerTableDeactivate dpt_deact;) | |
| 2876 | |
| 2877 // Clear any marks from a previous round | |
| 2878 verification_mark_bm()->clear_all(); | |
| 2879 assert(verification_mark_stack()->isEmpty(), "markStack should be empty"); | |
|
1145
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
2880 verify_work_stacks_empty(); |
| 0 | 2881 |
| 2882 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 2883 gch->ensure_parsability(false); // fill TLABs, but no need to retire them | |
| 2884 // Update the saved marks which may affect the root scans. | |
| 2885 gch->save_marks(); | |
| 2886 | |
| 2887 if (CMSRemarkVerifyVariant == 1) { | |
| 2888 // In this first variant of verification, we complete | |
| 2889 // all marking, then check if the new marks-verctor is | |
| 2890 // a subset of the CMS marks-vector. | |
| 2891 verify_after_remark_work_1(); | |
| 2892 } else if (CMSRemarkVerifyVariant == 2) { | |
| 2893 // In this second variant of verification, we flag an error | |
| 2894 // (i.e. an object reachable in the new marks-vector not reachable | |
| 2895 // in the CMS marks-vector) immediately, also indicating the | |
| 2896 // identify of an object (A) that references the unmarked object (B) -- | |
| 2897 // presumably, a mutation to A failed to be picked up by preclean/remark? | |
| 2898 verify_after_remark_work_2(); | |
| 2899 } else { | |
| 2900 warning("Unrecognized value %d for CMSRemarkVerifyVariant", | |
| 2901 CMSRemarkVerifyVariant); | |
| 2902 } | |
| 2903 gclog_or_tty->print(" done] "); | |
| 2904 return true; | |
| 2905 } | |
| 2906 | |
| 2907 void CMSCollector::verify_after_remark_work_1() { | |
| 2908 ResourceMark rm; | |
| 2909 HandleMark hm; | |
| 2910 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 2911 | |
| 2912 // Mark from roots one level into CMS | |
|
994
753cf9794df9
6885169: merge of 4957990 and 6863023 causes conflict on do_nmethods
jrose
parents:
993
diff
changeset
|
2913 MarkRefsIntoClosure notOlder(_span, verification_mark_bm()); |
| 0 | 2914 gch->rem_set()->prepare_for_younger_refs_iterate(false); // Not parallel. |
| 2915 | |
| 2916 gch->gen_process_strong_roots(_cmsGen->level(), | |
| 2917 true, // younger gens are roots | |
|
989
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
2918 true, // activate StrongRootsScope |
| 0 | 2919 true, // collecting perm gen |
| 2920 SharedHeap::ScanningOption(roots_scanning_options()), | |
|
989
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
2921 ¬Older, |
|
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
2922 true, // walk code active on stacks |
|
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
2923 NULL); |
| 0 | 2924 |
| 2925 // Now mark from the roots | |
| 2926 assert(_revisitStack.isEmpty(), "Should be empty"); | |
| 2927 MarkFromRootsClosure markFromRootsClosure(this, _span, | |
| 2928 verification_mark_bm(), verification_mark_stack(), &_revisitStack, | |
| 2929 false /* don't yield */, true /* verifying */); | |
| 2930 assert(_restart_addr == NULL, "Expected pre-condition"); | |
| 2931 verification_mark_bm()->iterate(&markFromRootsClosure); | |
| 2932 while (_restart_addr != NULL) { | |
| 2933 // Deal with stack overflow: by restarting at the indicated | |
| 2934 // address. | |
| 2935 HeapWord* ra = _restart_addr; | |
| 2936 markFromRootsClosure.reset(ra); | |
| 2937 _restart_addr = NULL; | |
| 2938 verification_mark_bm()->iterate(&markFromRootsClosure, ra, _span.end()); | |
| 2939 } | |
| 2940 assert(verification_mark_stack()->isEmpty(), "Should have been drained"); | |
| 2941 verify_work_stacks_empty(); | |
| 2942 // Should reset the revisit stack above, since no class tree | |
| 2943 // surgery is forthcoming. | |
| 2944 _revisitStack.reset(); // throwing away all contents | |
| 2945 | |
| 2946 // Marking completed -- now verify that each bit marked in | |
| 2947 // verification_mark_bm() is also marked in markBitMap(); flag all | |
| 2948 // errors by printing corresponding objects. | |
| 2949 VerifyMarkedClosure vcl(markBitMap()); | |
| 2950 verification_mark_bm()->iterate(&vcl); | |
| 2951 if (vcl.failed()) { | |
| 2952 gclog_or_tty->print("Verification failed"); | |
|
1145
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
2953 Universe::heap()->print_on(gclog_or_tty); |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
2954 fatal("CMS: failed marking verification after remark"); |
| 0 | 2955 } |
| 2956 } | |
| 2957 | |
| 2958 void CMSCollector::verify_after_remark_work_2() { | |
| 2959 ResourceMark rm; | |
| 2960 HandleMark hm; | |
| 2961 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 2962 | |
| 2963 // Mark from roots one level into CMS | |
| 2964 MarkRefsIntoVerifyClosure notOlder(_span, verification_mark_bm(), | |
|
994
753cf9794df9
6885169: merge of 4957990 and 6863023 causes conflict on do_nmethods
jrose
parents:
993
diff
changeset
|
2965 markBitMap()); |
| 0 | 2966 gch->rem_set()->prepare_for_younger_refs_iterate(false); // Not parallel. |
| 2967 gch->gen_process_strong_roots(_cmsGen->level(), | |
| 2968 true, // younger gens are roots | |
|
989
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
2969 true, // activate StrongRootsScope |
| 0 | 2970 true, // collecting perm gen |
| 2971 SharedHeap::ScanningOption(roots_scanning_options()), | |
|
989
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
2972 ¬Older, |
|
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
2973 true, // walk code active on stacks |
|
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
2974 NULL); |
| 0 | 2975 |
| 2976 // Now mark from the roots | |
| 2977 assert(_revisitStack.isEmpty(), "Should be empty"); | |
| 2978 MarkFromRootsVerifyClosure markFromRootsClosure(this, _span, | |
| 2979 verification_mark_bm(), markBitMap(), verification_mark_stack()); | |
| 2980 assert(_restart_addr == NULL, "Expected pre-condition"); | |
| 2981 verification_mark_bm()->iterate(&markFromRootsClosure); | |
| 2982 while (_restart_addr != NULL) { | |
| 2983 // Deal with stack overflow: by restarting at the indicated | |
| 2984 // address. | |
| 2985 HeapWord* ra = _restart_addr; | |
| 2986 markFromRootsClosure.reset(ra); | |
| 2987 _restart_addr = NULL; | |
| 2988 verification_mark_bm()->iterate(&markFromRootsClosure, ra, _span.end()); | |
| 2989 } | |
| 2990 assert(verification_mark_stack()->isEmpty(), "Should have been drained"); | |
| 2991 verify_work_stacks_empty(); | |
| 2992 // Should reset the revisit stack above, since no class tree | |
| 2993 // surgery is forthcoming. | |
| 2994 _revisitStack.reset(); // throwing away all contents | |
| 2995 | |
| 2996 // Marking completed -- now verify that each bit marked in | |
| 2997 // verification_mark_bm() is also marked in markBitMap(); flag all | |
| 2998 // errors by printing corresponding objects. | |
| 2999 VerifyMarkedClosure vcl(markBitMap()); | |
| 3000 verification_mark_bm()->iterate(&vcl); | |
| 3001 assert(!vcl.failed(), "Else verification above should not have succeeded"); | |
| 3002 } | |
| 3003 | |
| 3004 void ConcurrentMarkSweepGeneration::save_marks() { | |
| 3005 // delegate to CMS space | |
| 3006 cmsSpace()->save_marks(); | |
| 3007 for (uint i = 0; i < ParallelGCThreads; i++) { | |
| 3008 _par_gc_thread_states[i]->promo.startTrackingPromotions(); | |
| 3009 } | |
| 3010 } | |
| 3011 | |
| 3012 bool ConcurrentMarkSweepGeneration::no_allocs_since_save_marks() { | |
| 3013 return cmsSpace()->no_allocs_since_save_marks(); | |
| 3014 } | |
| 3015 | |
| 3016 #define CMS_SINCE_SAVE_MARKS_DEFN(OopClosureType, nv_suffix) \ | |
| 3017 \ | |
| 3018 void ConcurrentMarkSweepGeneration:: \ | |
| 3019 oop_since_save_marks_iterate##nv_suffix(OopClosureType* cl) { \ | |
| 3020 cl->set_generation(this); \ | |
| 3021 cmsSpace()->oop_since_save_marks_iterate##nv_suffix(cl); \ | |
| 3022 cl->reset_generation(); \ | |
| 3023 save_marks(); \ | |
| 3024 } | |
| 3025 | |
| 3026 ALL_SINCE_SAVE_MARKS_CLOSURES(CMS_SINCE_SAVE_MARKS_DEFN) | |
| 3027 | |
| 3028 void | |
| 3029 ConcurrentMarkSweepGeneration::object_iterate_since_last_GC(ObjectClosure* blk) | |
| 3030 { | |
| 3031 // Not currently implemented; need to do the following. -- ysr. | |
| 3032 // dld -- I think that is used for some sort of allocation profiler. So it | |
| 3033 // really means the objects allocated by the mutator since the last | |
| 3034 // GC. We could potentially implement this cheaply by recording only | |
| 3035 // the direct allocations in a side data structure. | |
| 3036 // | |
| 3037 // I think we probably ought not to be required to support these | |
| 3038 // iterations at any arbitrary point; I think there ought to be some | |
| 3039 // call to enable/disable allocation profiling in a generation/space, | |
| 3040 // and the iterator ought to return the objects allocated in the | |
| 3041 // gen/space since the enable call, or the last iterator call (which | |
| 3042 // will probably be at a GC.) That way, for gens like CM&S that would | |
| 3043 // require some extra data structure to support this, we only pay the | |
| 3044 // cost when it's in use... | |
| 3045 cmsSpace()->object_iterate_since_last_GC(blk); | |
| 3046 } | |
| 3047 | |
| 3048 void | |
| 3049 ConcurrentMarkSweepGeneration::younger_refs_iterate(OopsInGenClosure* cl) { | |
| 3050 cl->set_generation(this); | |
| 3051 younger_refs_in_space_iterate(_cmsSpace, cl); | |
| 3052 cl->reset_generation(); | |
| 3053 } | |
| 3054 | |
| 3055 void | |
| 3056 ConcurrentMarkSweepGeneration::oop_iterate(MemRegion mr, OopClosure* cl) { | |
| 3057 if (freelistLock()->owned_by_self()) { | |
| 3058 Generation::oop_iterate(mr, cl); | |
| 3059 } else { | |
| 3060 MutexLockerEx x(freelistLock(), Mutex::_no_safepoint_check_flag); | |
| 3061 Generation::oop_iterate(mr, cl); | |
| 3062 } | |
| 3063 } | |
| 3064 | |
| 3065 void | |
| 3066 ConcurrentMarkSweepGeneration::oop_iterate(OopClosure* cl) { | |
| 3067 if (freelistLock()->owned_by_self()) { | |
| 3068 Generation::oop_iterate(cl); | |
| 3069 } else { | |
| 3070 MutexLockerEx x(freelistLock(), Mutex::_no_safepoint_check_flag); | |
| 3071 Generation::oop_iterate(cl); | |
| 3072 } | |
| 3073 } | |
| 3074 | |
| 3075 void | |
| 3076 ConcurrentMarkSweepGeneration::object_iterate(ObjectClosure* cl) { | |
| 3077 if (freelistLock()->owned_by_self()) { | |
| 3078 Generation::object_iterate(cl); | |
| 3079 } else { | |
| 3080 MutexLockerEx x(freelistLock(), Mutex::_no_safepoint_check_flag); | |
| 3081 Generation::object_iterate(cl); | |
| 3082 } | |
| 3083 } | |
| 3084 | |
| 3085 void | |
|
517
e9be0e04635a
6689653: JMapPerm fails with UseConcMarkSweepIncGC and compressed oops off
jmasa
parents:
457
diff
changeset
|
3086 ConcurrentMarkSweepGeneration::safe_object_iterate(ObjectClosure* cl) { |
|
e9be0e04635a
6689653: JMapPerm fails with UseConcMarkSweepIncGC and compressed oops off
jmasa
parents:
457
diff
changeset
|
3087 if (freelistLock()->owned_by_self()) { |
|
e9be0e04635a
6689653: JMapPerm fails with UseConcMarkSweepIncGC and compressed oops off
jmasa
parents:
457
diff
changeset
|
3088 Generation::safe_object_iterate(cl); |
|
e9be0e04635a
6689653: JMapPerm fails with UseConcMarkSweepIncGC and compressed oops off
jmasa
parents:
457
diff
changeset
|
3089 } else { |
|
e9be0e04635a
6689653: JMapPerm fails with UseConcMarkSweepIncGC and compressed oops off
jmasa
parents:
457
diff
changeset
|
3090 MutexLockerEx x(freelistLock(), Mutex::_no_safepoint_check_flag); |
|
e9be0e04635a
6689653: JMapPerm fails with UseConcMarkSweepIncGC and compressed oops off
jmasa
parents:
457
diff
changeset
|
3091 Generation::safe_object_iterate(cl); |
|
e9be0e04635a
6689653: JMapPerm fails with UseConcMarkSweepIncGC and compressed oops off
jmasa
parents:
457
diff
changeset
|
3092 } |
|
e9be0e04635a
6689653: JMapPerm fails with UseConcMarkSweepIncGC and compressed oops off
jmasa
parents:
457
diff
changeset
|
3093 } |
|
e9be0e04635a
6689653: JMapPerm fails with UseConcMarkSweepIncGC and compressed oops off
jmasa
parents:
457
diff
changeset
|
3094 |
|
e9be0e04635a
6689653: JMapPerm fails with UseConcMarkSweepIncGC and compressed oops off
jmasa
parents:
457
diff
changeset
|
3095 void |
| 0 | 3096 ConcurrentMarkSweepGeneration::pre_adjust_pointers() { |
| 3097 } | |
| 3098 | |
| 3099 void | |
| 3100 ConcurrentMarkSweepGeneration::post_compact() { | |
| 3101 } | |
| 3102 | |
| 3103 void | |
| 3104 ConcurrentMarkSweepGeneration::prepare_for_verify() { | |
| 3105 // Fix the linear allocation blocks to look like free blocks. | |
| 3106 | |
| 3107 // Locks are normally acquired/released in gc_prologue/gc_epilogue, but those | |
| 3108 // are not called when the heap is verified during universe initialization and | |
| 3109 // at vm shutdown. | |
| 3110 if (freelistLock()->owned_by_self()) { | |
| 3111 cmsSpace()->prepare_for_verify(); | |
| 3112 } else { | |
| 3113 MutexLockerEx fll(freelistLock(), Mutex::_no_safepoint_check_flag); | |
| 3114 cmsSpace()->prepare_for_verify(); | |
| 3115 } | |
| 3116 } | |
| 3117 | |
| 3118 void | |
| 3119 ConcurrentMarkSweepGeneration::verify(bool allow_dirty /* ignored */) { | |
| 3120 // Locks are normally acquired/released in gc_prologue/gc_epilogue, but those | |
| 3121 // are not called when the heap is verified during universe initialization and | |
| 3122 // at vm shutdown. | |
| 3123 if (freelistLock()->owned_by_self()) { | |
| 3124 cmsSpace()->verify(false /* ignored */); | |
| 3125 } else { | |
| 3126 MutexLockerEx fll(freelistLock(), Mutex::_no_safepoint_check_flag); | |
| 3127 cmsSpace()->verify(false /* ignored */); | |
| 3128 } | |
| 3129 } | |
| 3130 | |
| 3131 void CMSCollector::verify(bool allow_dirty /* ignored */) { | |
| 3132 _cmsGen->verify(allow_dirty); | |
| 3133 _permGen->verify(allow_dirty); | |
| 3134 } | |
| 3135 | |
| 3136 #ifndef PRODUCT | |
| 3137 bool CMSCollector::overflow_list_is_empty() const { | |
| 3138 assert(_num_par_pushes >= 0, "Inconsistency"); | |
| 3139 if (_overflow_list == NULL) { | |
| 3140 assert(_num_par_pushes == 0, "Inconsistency"); | |
| 3141 } | |
| 3142 return _overflow_list == NULL; | |
| 3143 } | |
| 3144 | |
| 3145 // The methods verify_work_stacks_empty() and verify_overflow_empty() | |
| 3146 // merely consolidate assertion checks that appear to occur together frequently. | |
| 3147 void CMSCollector::verify_work_stacks_empty() const { | |
| 3148 assert(_markStack.isEmpty(), "Marking stack should be empty"); | |
| 3149 assert(overflow_list_is_empty(), "Overflow list should be empty"); | |
| 3150 } | |
| 3151 | |
| 3152 void CMSCollector::verify_overflow_empty() const { | |
| 3153 assert(overflow_list_is_empty(), "Overflow list should be empty"); | |
| 3154 assert(no_preserved_marks(), "No preserved marks"); | |
| 3155 } | |
| 3156 #endif // PRODUCT | |
| 3157 | |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3158 // Decide if we want to enable class unloading as part of the |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3159 // ensuing concurrent GC cycle. We will collect the perm gen and |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3160 // unload classes if it's the case that: |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3161 // (1) an explicit gc request has been made and the flag |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3162 // ExplicitGCInvokesConcurrentAndUnloadsClasses is set, OR |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3163 // (2) (a) class unloading is enabled at the command line, and |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3164 // (b) (i) perm gen threshold has been crossed, or |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3165 // (ii) old gen is getting really full, or |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3166 // (iii) the previous N CMS collections did not collect the |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3167 // perm gen |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3168 // NOTE: Provided there is no change in the state of the heap between |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3169 // calls to this method, it should have idempotent results. Moreover, |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3170 // its results should be monotonically increasing (i.e. going from 0 to 1, |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3171 // but not 1 to 0) between successive calls between which the heap was |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3172 // not collected. For the implementation below, it must thus rely on |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3173 // the property that concurrent_cycles_since_last_unload() |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3174 // will not decrease unless a collection cycle happened and that |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3175 // _permGen->should_concurrent_collect() and _cmsGen->is_too_full() are |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3176 // themselves also monotonic in that sense. See check_monotonicity() |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3177 // below. |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3178 bool CMSCollector::update_should_unload_classes() { |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3179 _should_unload_classes = false; |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3180 // Condition 1 above |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3181 if (_full_gc_requested && ExplicitGCInvokesConcurrentAndUnloadsClasses) { |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3182 _should_unload_classes = true; |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3183 } else if (CMSClassUnloadingEnabled) { // Condition 2.a above |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3184 // Disjuncts 2.b.(i,ii,iii) above |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3185 _should_unload_classes = (concurrent_cycles_since_last_unload() >= |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3186 CMSClassUnloadingMaxInterval) |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3187 || _permGen->should_concurrent_collect() |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3188 || _cmsGen->is_too_full(); |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3189 } |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3190 return _should_unload_classes; |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3191 } |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3192 |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3193 bool ConcurrentMarkSweepGeneration::is_too_full() const { |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3194 bool res = should_concurrent_collect(); |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3195 res = res && (occupancy() > (double)CMSIsTooFullPercentage/100.0); |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3196 return res; |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3197 } |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3198 |
| 0 | 3199 void CMSCollector::setup_cms_unloading_and_verification_state() { |
| 3200 const bool should_verify = VerifyBeforeGC || VerifyAfterGC || VerifyDuringGC | |
| 3201 || VerifyBeforeExit; | |
| 3202 const int rso = SharedHeap::SO_Symbols | SharedHeap::SO_Strings | |
| 3203 | SharedHeap::SO_CodeCache; | |
| 3204 | |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3205 if (should_unload_classes()) { // Should unload classes this cycle |
| 0 | 3206 remove_root_scanning_option(rso); // Shrink the root set appropriately |
| 3207 set_verifying(should_verify); // Set verification state for this cycle | |
| 3208 return; // Nothing else needs to be done at this time | |
| 3209 } | |
| 3210 | |
| 3211 // Not unloading classes this cycle | |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3212 assert(!should_unload_classes(), "Inconsitency!"); |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
3213 if ((!verifying() || unloaded_classes_last_cycle()) && should_verify) { |
| 0 | 3214 // We were not verifying, or we _were_ unloading classes in the last cycle, |
| 3215 // AND some verification options are enabled this cycle; in this case, | |
| 3216 // we must make sure that the deadness map is allocated if not already so, | |
| 3217 // and cleared (if already allocated previously -- | |
| 3218 // CMSBitMap::sizeInBits() is used to determine if it's allocated). | |
| 3219 if (perm_gen_verify_bit_map()->sizeInBits() == 0) { | |
| 3220 if (!perm_gen_verify_bit_map()->allocate(_permGen->reserved())) { | |
| 3221 warning("Failed to allocate permanent generation verification CMS Bit Map;\n" | |
| 3222 "permanent generation verification disabled"); | |
| 3223 return; // Note that we leave verification disabled, so we'll retry this | |
| 3224 // allocation next cycle. We _could_ remember this failure | |
| 3225 // and skip further attempts and permanently disable verification | |
| 3226 // attempts if that is considered more desirable. | |
| 3227 } | |
| 3228 assert(perm_gen_verify_bit_map()->covers(_permGen->reserved()), | |
| 3229 "_perm_gen_ver_bit_map inconsistency?"); | |
| 3230 } else { | |
| 3231 perm_gen_verify_bit_map()->clear_all(); | |
| 3232 } | |
| 3233 // Include symbols, strings and code cache elements to prevent their resurrection. | |
| 3234 add_root_scanning_option(rso); | |
| 3235 set_verifying(true); | |
| 3236 } else if (verifying() && !should_verify) { | |
| 3237 // We were verifying, but some verification flags got disabled. | |
| 3238 set_verifying(false); | |
| 3239 // Exclude symbols, strings and code cache elements from root scanning to | |
| 3240 // reduce IM and RM pauses. | |
| 3241 remove_root_scanning_option(rso); | |
| 3242 } | |
| 3243 } | |
| 3244 | |
| 3245 | |
| 3246 #ifndef PRODUCT | |
| 3247 HeapWord* CMSCollector::block_start(const void* p) const { | |
| 3248 const HeapWord* addr = (HeapWord*)p; | |
| 3249 if (_span.contains(p)) { | |
| 3250 if (_cmsGen->cmsSpace()->is_in_reserved(addr)) { | |
| 3251 return _cmsGen->cmsSpace()->block_start(p); | |
| 3252 } else { | |
| 3253 assert(_permGen->cmsSpace()->is_in_reserved(addr), | |
| 3254 "Inconsistent _span?"); | |
| 3255 return _permGen->cmsSpace()->block_start(p); | |
| 3256 } | |
| 3257 } | |
| 3258 return NULL; | |
| 3259 } | |
| 3260 #endif | |
| 3261 | |
| 3262 HeapWord* | |
| 3263 ConcurrentMarkSweepGeneration::expand_and_allocate(size_t word_size, | |
| 3264 bool tlab, | |
| 3265 bool parallel) { | |
| 3266 assert(!tlab, "Can't deal with TLAB allocation"); | |
| 3267 MutexLockerEx x(freelistLock(), Mutex::_no_safepoint_check_flag); | |
| 3268 expand(word_size*HeapWordSize, MinHeapDeltaBytes, | |
| 3269 CMSExpansionCause::_satisfy_allocation); | |
| 3270 if (GCExpandToAllocateDelayMillis > 0) { | |
| 3271 os::sleep(Thread::current(), GCExpandToAllocateDelayMillis, false); | |
| 3272 } | |
|
9
173195ff483a
6642634: Test nsk/regression/b6186200 crashed with SIGSEGV
ysr
parents:
7
diff
changeset
|
3273 return have_lock_and_allocate(word_size, tlab); |
| 0 | 3274 } |
| 3275 | |
| 3276 // YSR: All of this generation expansion/shrinking stuff is an exact copy of | |
| 3277 // OneContigSpaceCardGeneration, which makes me wonder if we should move this | |
| 3278 // to CardGeneration and share it... | |
|
271
818a18cd69a8
6730514: assertion failure in mangling code when expanding by 0 bytes
jmasa
parents:
196
diff
changeset
|
3279 bool ConcurrentMarkSweepGeneration::expand(size_t bytes, size_t expand_bytes) { |
|
818a18cd69a8
6730514: assertion failure in mangling code when expanding by 0 bytes
jmasa
parents:
196
diff
changeset
|
3280 return CardGeneration::expand(bytes, expand_bytes); |
|
818a18cd69a8
6730514: assertion failure in mangling code when expanding by 0 bytes
jmasa
parents:
196
diff
changeset
|
3281 } |
|
818a18cd69a8
6730514: assertion failure in mangling code when expanding by 0 bytes
jmasa
parents:
196
diff
changeset
|
3282 |
| 0 | 3283 void ConcurrentMarkSweepGeneration::expand(size_t bytes, size_t expand_bytes, |
| 3284 CMSExpansionCause::Cause cause) | |
| 3285 { | |
|
271
818a18cd69a8
6730514: assertion failure in mangling code when expanding by 0 bytes
jmasa
parents:
196
diff
changeset
|
3286 |
|
818a18cd69a8
6730514: assertion failure in mangling code when expanding by 0 bytes
jmasa
parents:
196
diff
changeset
|
3287 bool success = expand(bytes, expand_bytes); |
|
818a18cd69a8
6730514: assertion failure in mangling code when expanding by 0 bytes
jmasa
parents:
196
diff
changeset
|
3288 |
| 0 | 3289 // remember why we expanded; this information is used |
| 3290 // by shouldConcurrentCollect() when making decisions on whether to start | |
| 3291 // a new CMS cycle. | |
| 3292 if (success) { | |
| 3293 set_expansion_cause(cause); | |
| 3294 if (PrintGCDetails && Verbose) { | |
| 3295 gclog_or_tty->print_cr("Expanded CMS gen for %s", | |
| 3296 CMSExpansionCause::to_string(cause)); | |
| 3297 } | |
| 3298 } | |
| 3299 } | |
| 3300 | |
| 3301 HeapWord* ConcurrentMarkSweepGeneration::expand_and_par_lab_allocate(CMSParGCThreadState* ps, size_t word_sz) { | |
| 3302 HeapWord* res = NULL; | |
| 3303 MutexLocker x(ParGCRareEvent_lock); | |
| 3304 while (true) { | |
| 3305 // Expansion by some other thread might make alloc OK now: | |
| 3306 res = ps->lab.alloc(word_sz); | |
| 3307 if (res != NULL) return res; | |
| 3308 // If there's not enough expansion space available, give up. | |
| 3309 if (_virtual_space.uncommitted_size() < (word_sz * HeapWordSize)) { | |
| 3310 return NULL; | |
| 3311 } | |
| 3312 // Otherwise, we try expansion. | |
| 3313 expand(word_sz*HeapWordSize, MinHeapDeltaBytes, | |
| 3314 CMSExpansionCause::_allocate_par_lab); | |
| 3315 // Now go around the loop and try alloc again; | |
| 3316 // A competing par_promote might beat us to the expansion space, | |
| 3317 // so we may go around the loop again if promotion fails agaion. | |
| 3318 if (GCExpandToAllocateDelayMillis > 0) { | |
| 3319 os::sleep(Thread::current(), GCExpandToAllocateDelayMillis, false); | |
| 3320 } | |
| 3321 } | |
| 3322 } | |
| 3323 | |
| 3324 | |
| 3325 bool ConcurrentMarkSweepGeneration::expand_and_ensure_spooling_space( | |
| 3326 PromotionInfo* promo) { | |
| 3327 MutexLocker x(ParGCRareEvent_lock); | |
| 3328 size_t refill_size_bytes = promo->refillSize() * HeapWordSize; | |
| 3329 while (true) { | |
| 3330 // Expansion by some other thread might make alloc OK now: | |
| 3331 if (promo->ensure_spooling_space()) { | |
| 3332 assert(promo->has_spooling_space(), | |
| 3333 "Post-condition of successful ensure_spooling_space()"); | |
| 3334 return true; | |
| 3335 } | |
| 3336 // If there's not enough expansion space available, give up. | |
| 3337 if (_virtual_space.uncommitted_size() < refill_size_bytes) { | |
| 3338 return false; | |
| 3339 } | |
| 3340 // Otherwise, we try expansion. | |
| 3341 expand(refill_size_bytes, MinHeapDeltaBytes, | |
| 3342 CMSExpansionCause::_allocate_par_spooling_space); | |
| 3343 // Now go around the loop and try alloc again; | |
| 3344 // A competing allocation might beat us to the expansion space, | |
| 3345 // so we may go around the loop again if allocation fails again. | |
| 3346 if (GCExpandToAllocateDelayMillis > 0) { | |
| 3347 os::sleep(Thread::current(), GCExpandToAllocateDelayMillis, false); | |
| 3348 } | |
| 3349 } | |
| 3350 } | |
| 3351 | |
| 3352 | |
| 3353 | |
| 3354 void ConcurrentMarkSweepGeneration::shrink(size_t bytes) { | |
| 3355 assert_locked_or_safepoint(Heap_lock); | |
| 3356 size_t size = ReservedSpace::page_align_size_down(bytes); | |
| 3357 if (size > 0) { | |
| 3358 shrink_by(size); | |
| 3359 } | |
| 3360 } | |
| 3361 | |
| 3362 bool ConcurrentMarkSweepGeneration::grow_by(size_t bytes) { | |
| 3363 assert_locked_or_safepoint(Heap_lock); | |
| 3364 bool result = _virtual_space.expand_by(bytes); | |
| 3365 if (result) { | |
| 3366 HeapWord* old_end = _cmsSpace->end(); | |
| 3367 size_t new_word_size = | |
| 3368 heap_word_size(_virtual_space.committed_size()); | |
| 3369 MemRegion mr(_cmsSpace->bottom(), new_word_size); | |
| 3370 _bts->resize(new_word_size); // resize the block offset shared array | |
| 3371 Universe::heap()->barrier_set()->resize_covered_region(mr); | |
| 3372 // Hmmmm... why doesn't CFLS::set_end verify locking? | |
| 3373 // This is quite ugly; FIX ME XXX | |
|
1145
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
3374 _cmsSpace->assert_locked(freelistLock()); |
| 0 | 3375 _cmsSpace->set_end((HeapWord*)_virtual_space.high()); |
| 3376 | |
| 3377 // update the space and generation capacity counters | |
| 3378 if (UsePerfData) { | |
| 3379 _space_counters->update_capacity(); | |
| 3380 _gen_counters->update_all(); | |
| 3381 } | |
| 3382 | |
| 3383 if (Verbose && PrintGC) { | |
| 3384 size_t new_mem_size = _virtual_space.committed_size(); | |
| 3385 size_t old_mem_size = new_mem_size - bytes; | |
| 3386 gclog_or_tty->print_cr("Expanding %s from %ldK by %ldK to %ldK", | |
| 3387 name(), old_mem_size/K, bytes/K, new_mem_size/K); | |
| 3388 } | |
| 3389 } | |
| 3390 return result; | |
| 3391 } | |
| 3392 | |
| 3393 bool ConcurrentMarkSweepGeneration::grow_to_reserved() { | |
| 3394 assert_locked_or_safepoint(Heap_lock); | |
| 3395 bool success = true; | |
| 3396 const size_t remaining_bytes = _virtual_space.uncommitted_size(); | |
| 3397 if (remaining_bytes > 0) { | |
| 3398 success = grow_by(remaining_bytes); | |
| 3399 DEBUG_ONLY(if (!success) warning("grow to reserved failed");) | |
| 3400 } | |
| 3401 return success; | |
| 3402 } | |
| 3403 | |
| 3404 void ConcurrentMarkSweepGeneration::shrink_by(size_t bytes) { | |
| 3405 assert_locked_or_safepoint(Heap_lock); | |
| 3406 assert_lock_strong(freelistLock()); | |
| 3407 // XXX Fix when compaction is implemented. | |
| 3408 warning("Shrinking of CMS not yet implemented"); | |
| 3409 return; | |
| 3410 } | |
| 3411 | |
| 3412 | |
| 3413 // Simple ctor/dtor wrapper for accounting & timer chores around concurrent | |
| 3414 // phases. | |
| 3415 class CMSPhaseAccounting: public StackObj { | |
| 3416 public: | |
| 3417 CMSPhaseAccounting(CMSCollector *collector, | |
| 3418 const char *phase, | |
| 3419 bool print_cr = true); | |
| 3420 ~CMSPhaseAccounting(); | |
| 3421 | |
| 3422 private: | |
| 3423 CMSCollector *_collector; | |
| 3424 const char *_phase; | |
| 3425 elapsedTimer _wallclock; | |
| 3426 bool _print_cr; | |
| 3427 | |
| 3428 public: | |
| 3429 // Not MT-safe; so do not pass around these StackObj's | |
| 3430 // where they may be accessed by other threads. | |
| 3431 jlong wallclock_millis() { | |
| 3432 assert(_wallclock.is_active(), "Wall clock should not stop"); | |
| 3433 _wallclock.stop(); // to record time | |
| 3434 jlong ret = _wallclock.milliseconds(); | |
| 3435 _wallclock.start(); // restart | |
| 3436 return ret; | |
| 3437 } | |
| 3438 }; | |
| 3439 | |
| 3440 CMSPhaseAccounting::CMSPhaseAccounting(CMSCollector *collector, | |
| 3441 const char *phase, | |
| 3442 bool print_cr) : | |
| 3443 _collector(collector), _phase(phase), _print_cr(print_cr) { | |
| 3444 | |
| 3445 if (PrintCMSStatistics != 0) { | |
| 3446 _collector->resetYields(); | |
| 3447 } | |
| 3448 if (PrintGCDetails && PrintGCTimeStamps) { | |
| 3449 gclog_or_tty->date_stamp(PrintGCDateStamps); | |
| 3450 gclog_or_tty->stamp(); | |
| 3451 gclog_or_tty->print_cr(": [%s-concurrent-%s-start]", | |
| 3452 _collector->cmsGen()->short_name(), _phase); | |
| 3453 } | |
| 3454 _collector->resetTimer(); | |
| 3455 _wallclock.start(); | |
| 3456 _collector->startTimer(); | |
| 3457 } | |
| 3458 | |
| 3459 CMSPhaseAccounting::~CMSPhaseAccounting() { | |
| 3460 assert(_wallclock.is_active(), "Wall clock should not have stopped"); | |
| 3461 _collector->stopTimer(); | |
| 3462 _wallclock.stop(); | |
| 3463 if (PrintGCDetails) { | |
| 3464 gclog_or_tty->date_stamp(PrintGCDateStamps); | |
| 3465 if (PrintGCTimeStamps) { | |
| 3466 gclog_or_tty->stamp(); | |
| 3467 gclog_or_tty->print(": "); | |
| 3468 } | |
| 3469 gclog_or_tty->print("[%s-concurrent-%s: %3.3f/%3.3f secs]", | |
| 3470 _collector->cmsGen()->short_name(), | |
| 3471 _phase, _collector->timerValue(), _wallclock.seconds()); | |
| 3472 if (_print_cr) { | |
| 3473 gclog_or_tty->print_cr(""); | |
| 3474 } | |
| 3475 if (PrintCMSStatistics != 0) { | |
| 3476 gclog_or_tty->print_cr(" (CMS-concurrent-%s yielded %d times)", _phase, | |
| 3477 _collector->yields()); | |
| 3478 } | |
| 3479 } | |
| 3480 } | |
| 3481 | |
| 3482 // CMS work | |
| 3483 | |
| 3484 // Checkpoint the roots into this generation from outside | |
| 3485 // this generation. [Note this initial checkpoint need only | |
| 3486 // be approximate -- we'll do a catch up phase subsequently.] | |
| 3487 void CMSCollector::checkpointRootsInitial(bool asynch) { | |
| 3488 assert(_collectorState == InitialMarking, "Wrong collector state"); | |
| 3489 check_correct_thread_executing(); | |
|
1703
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
3490 TraceCMSMemoryManagerStats tms(_collectorState); |
| 0 | 3491 ReferenceProcessor* rp = ref_processor(); |
| 3492 SpecializationStats::clear(); | |
| 3493 assert(_restart_addr == NULL, "Control point invariant"); | |
| 3494 if (asynch) { | |
| 3495 // acquire locks for subsequent manipulations | |
| 3496 MutexLockerEx x(bitMapLock(), | |
| 3497 Mutex::_no_safepoint_check_flag); | |
| 3498 checkpointRootsInitialWork(asynch); | |
| 3499 rp->verify_no_references_recorded(); | |
| 3500 rp->enable_discovery(); // enable ("weak") refs discovery | |
| 3501 _collectorState = Marking; | |
| 3502 } else { | |
| 3503 // (Weak) Refs discovery: this is controlled from genCollectedHeap::do_collection | |
| 3504 // which recognizes if we are a CMS generation, and doesn't try to turn on | |
| 3505 // discovery; verify that they aren't meddling. | |
| 3506 assert(!rp->discovery_is_atomic(), | |
| 3507 "incorrect setting of discovery predicate"); | |
| 3508 assert(!rp->discovery_enabled(), "genCollectedHeap shouldn't control " | |
| 3509 "ref discovery for this generation kind"); | |
| 3510 // already have locks | |
| 3511 checkpointRootsInitialWork(asynch); | |
| 3512 rp->enable_discovery(); // now enable ("weak") refs discovery | |
| 3513 _collectorState = Marking; | |
| 3514 } | |
| 3515 SpecializationStats::print(); | |
| 3516 } | |
| 3517 | |
| 3518 void CMSCollector::checkpointRootsInitialWork(bool asynch) { | |
| 3519 assert(SafepointSynchronize::is_at_safepoint(), "world should be stopped"); | |
| 3520 assert(_collectorState == InitialMarking, "just checking"); | |
| 3521 | |
| 3522 // If there has not been a GC[n-1] since last GC[n] cycle completed, | |
| 3523 // precede our marking with a collection of all | |
| 3524 // younger generations to keep floating garbage to a minimum. | |
| 3525 // XXX: we won't do this for now -- it's an optimization to be done later. | |
| 3526 | |
| 3527 // already have locks | |
| 3528 assert_lock_strong(bitMapLock()); | |
| 3529 assert(_markBitMap.isAllClear(), "was reset at end of previous cycle"); | |
| 3530 | |
| 3531 // Setup the verification and class unloading state for this | |
| 3532 // CMS collection cycle. | |
| 3533 setup_cms_unloading_and_verification_state(); | |
| 3534 | |
| 3535 NOT_PRODUCT(TraceTime t("\ncheckpointRootsInitialWork", | |
| 3536 PrintGCDetails && Verbose, true, gclog_or_tty);) | |
| 3537 if (UseAdaptiveSizePolicy) { | |
| 3538 size_policy()->checkpoint_roots_initial_begin(); | |
| 3539 } | |
| 3540 | |
| 3541 // Reset all the PLAB chunk arrays if necessary. | |
| 3542 if (_survivor_plab_array != NULL && !CMSPLABRecordAlways) { | |
| 3543 reset_survivor_plab_arrays(); | |
| 3544 } | |
| 3545 | |
| 3546 ResourceMark rm; | |
| 3547 HandleMark hm; | |
| 3548 | |
| 3549 FalseClosure falseClosure; | |
| 3550 // In the case of a synchronous collection, we will elide the | |
| 3551 // remark step, so it's important to catch all the nmethod oops | |
|
994
753cf9794df9
6885169: merge of 4957990 and 6863023 causes conflict on do_nmethods
jrose
parents:
993
diff
changeset
|
3552 // in this step. |
|
753cf9794df9
6885169: merge of 4957990 and 6863023 causes conflict on do_nmethods
jrose
parents:
993
diff
changeset
|
3553 // The final 'true' flag to gen_process_strong_roots will ensure this. |
|
753cf9794df9
6885169: merge of 4957990 and 6863023 causes conflict on do_nmethods
jrose
parents:
993
diff
changeset
|
3554 // If 'async' is true, we can relax the nmethod tracing. |
|
753cf9794df9
6885169: merge of 4957990 and 6863023 causes conflict on do_nmethods
jrose
parents:
993
diff
changeset
|
3555 MarkRefsIntoClosure notOlder(_span, &_markBitMap); |
| 0 | 3556 GenCollectedHeap* gch = GenCollectedHeap::heap(); |
| 3557 | |
| 3558 verify_work_stacks_empty(); | |
| 3559 verify_overflow_empty(); | |
| 3560 | |
| 3561 gch->ensure_parsability(false); // fill TLABs, but no need to retire them | |
| 3562 // Update the saved marks which may affect the root scans. | |
| 3563 gch->save_marks(); | |
| 3564 | |
| 3565 // weak reference processing has not started yet. | |
| 3566 ref_processor()->set_enqueuing_is_done(false); | |
| 3567 | |
| 3568 { | |
| 935 | 3569 // This is not needed. DEBUG_ONLY(RememberKlassesChecker imx(true);) |
| 0 | 3570 COMPILER2_PRESENT(DerivedPointerTableDeactivate dpt_deact;) |
| 3571 gch->rem_set()->prepare_for_younger_refs_iterate(false); // Not parallel. | |
| 3572 gch->gen_process_strong_roots(_cmsGen->level(), | |
| 3573 true, // younger gens are roots | |
|
989
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
3574 true, // activate StrongRootsScope |
| 0 | 3575 true, // collecting perm gen |
| 3576 SharedHeap::ScanningOption(roots_scanning_options()), | |
|
989
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
3577 ¬Older, |
|
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
3578 true, // walk all of code cache if (so & SO_CodeCache) |
|
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
3579 NULL); |
| 0 | 3580 } |
| 3581 | |
| 3582 // Clear mod-union table; it will be dirtied in the prologue of | |
| 3583 // CMS generation per each younger generation collection. | |
| 3584 | |
| 3585 assert(_modUnionTable.isAllClear(), | |
| 3586 "Was cleared in most recent final checkpoint phase" | |
| 3587 " or no bits are set in the gc_prologue before the start of the next " | |
| 3588 "subsequent marking phase."); | |
| 3589 | |
| 3590 // Temporarily disabled, since pre/post-consumption closures don't | |
| 3591 // care about precleaned cards | |
| 3592 #if 0 | |
| 3593 { | |
| 3594 MemRegion mr = MemRegion((HeapWord*)_virtual_space.low(), | |
| 3595 (HeapWord*)_virtual_space.high()); | |
| 3596 _ct->ct_bs()->preclean_dirty_cards(mr); | |
| 3597 } | |
| 3598 #endif | |
| 3599 | |
| 3600 // Save the end of the used_region of the constituent generations | |
| 3601 // to be used to limit the extent of sweep in each generation. | |
| 3602 save_sweep_limits(); | |
| 3603 if (UseAdaptiveSizePolicy) { | |
| 3604 size_policy()->checkpoint_roots_initial_end(gch->gc_cause()); | |
| 3605 } | |
| 3606 verify_overflow_empty(); | |
| 3607 } | |
| 3608 | |
| 3609 bool CMSCollector::markFromRoots(bool asynch) { | |
| 3610 // we might be tempted to assert that: | |
| 3611 // assert(asynch == !SafepointSynchronize::is_at_safepoint(), | |
| 3612 // "inconsistent argument?"); | |
| 3613 // However that wouldn't be right, because it's possible that | |
| 3614 // a safepoint is indeed in progress as a younger generation | |
| 3615 // stop-the-world GC happens even as we mark in this generation. | |
| 3616 assert(_collectorState == Marking, "inconsistent state?"); | |
| 3617 check_correct_thread_executing(); | |
| 3618 verify_overflow_empty(); | |
| 3619 | |
| 3620 bool res; | |
| 3621 if (asynch) { | |
| 3622 | |
| 3623 // Start the timers for adaptive size policy for the concurrent phases | |
| 3624 // Do it here so that the foreground MS can use the concurrent | |
| 3625 // timer since a foreground MS might has the sweep done concurrently | |
| 3626 // or STW. | |
| 3627 if (UseAdaptiveSizePolicy) { | |
| 3628 size_policy()->concurrent_marking_begin(); | |
| 3629 } | |
| 3630 | |
| 3631 // Weak ref discovery note: We may be discovering weak | |
| 3632 // refs in this generation concurrent (but interleaved) with | |
| 3633 // weak ref discovery by a younger generation collector. | |
| 3634 | |
| 3635 CMSTokenSyncWithLocks ts(true, bitMapLock()); | |
| 3636 TraceCPUTime tcpu(PrintGCDetails, true, gclog_or_tty); | |
| 3637 CMSPhaseAccounting pa(this, "mark", !PrintGCDetails); | |
| 3638 res = markFromRootsWork(asynch); | |
| 3639 if (res) { | |
| 3640 _collectorState = Precleaning; | |
| 3641 } else { // We failed and a foreground collection wants to take over | |
| 3642 assert(_foregroundGCIsActive, "internal state inconsistency"); | |
| 3643 assert(_restart_addr == NULL, "foreground will restart from scratch"); | |
| 3644 if (PrintGCDetails) { | |
| 3645 gclog_or_tty->print_cr("bailing out to foreground collection"); | |
| 3646 } | |
| 3647 } | |
| 3648 if (UseAdaptiveSizePolicy) { | |
| 3649 size_policy()->concurrent_marking_end(); | |
| 3650 } | |
| 3651 } else { | |
| 3652 assert(SafepointSynchronize::is_at_safepoint(), | |
| 3653 "inconsistent with asynch == false"); | |
| 3654 if (UseAdaptiveSizePolicy) { | |
| 3655 size_policy()->ms_collection_marking_begin(); | |
| 3656 } | |
| 3657 // already have locks | |
| 3658 res = markFromRootsWork(asynch); | |
| 3659 _collectorState = FinalMarking; | |
| 3660 if (UseAdaptiveSizePolicy) { | |
| 3661 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 3662 size_policy()->ms_collection_marking_end(gch->gc_cause()); | |
| 3663 } | |
| 3664 } | |
| 3665 verify_overflow_empty(); | |
| 3666 return res; | |
| 3667 } | |
| 3668 | |
| 3669 bool CMSCollector::markFromRootsWork(bool asynch) { | |
| 3670 // iterate over marked bits in bit map, doing a full scan and mark | |
| 3671 // from these roots using the following algorithm: | |
| 3672 // . if oop is to the right of the current scan pointer, | |
| 3673 // mark corresponding bit (we'll process it later) | |
| 3674 // . else (oop is to left of current scan pointer) | |
| 3675 // push oop on marking stack | |
| 3676 // . drain the marking stack | |
| 3677 | |
| 3678 // Note that when we do a marking step we need to hold the | |
| 3679 // bit map lock -- recall that direct allocation (by mutators) | |
| 3680 // and promotion (by younger generation collectors) is also | |
| 3681 // marking the bit map. [the so-called allocate live policy.] | |
| 3682 // Because the implementation of bit map marking is not | |
| 3683 // robust wrt simultaneous marking of bits in the same word, | |
| 3684 // we need to make sure that there is no such interference | |
| 3685 // between concurrent such updates. | |
| 3686 | |
| 3687 // already have locks | |
| 3688 assert_lock_strong(bitMapLock()); | |
| 3689 | |
| 3690 // Clear the revisit stack, just in case there are any | |
| 3691 // obsolete contents from a short-circuited previous CMS cycle. | |
| 3692 _revisitStack.reset(); | |
| 3693 verify_work_stacks_empty(); | |
| 3694 verify_overflow_empty(); | |
| 3695 assert(_revisitStack.isEmpty(), "tabula rasa"); | |
|
1190
4788266644c1
6895236: CMS: cmsOopClosures.inline.hpp:43 assert(..., "Should remember klasses in this context")
jmasa
parents:
1145
diff
changeset
|
3696 DEBUG_ONLY(RememberKlassesChecker cmx(should_unload_classes());) |
| 0 | 3697 bool result = false; |
| 1284 | 3698 if (CMSConcurrentMTEnabled && ConcGCThreads > 0) { |
| 0 | 3699 result = do_marking_mt(asynch); |
| 3700 } else { | |
| 3701 result = do_marking_st(asynch); | |
| 3702 } | |
| 3703 return result; | |
| 3704 } | |
| 3705 | |
| 3706 // Forward decl | |
| 3707 class CMSConcMarkingTask; | |
| 3708 | |
| 3709 class CMSConcMarkingTerminator: public ParallelTaskTerminator { | |
| 3710 CMSCollector* _collector; | |
| 3711 CMSConcMarkingTask* _task; | |
| 3712 bool _yield; | |
| 3713 protected: | |
| 3714 virtual void yield(); | |
| 3715 public: | |
| 3716 // "n_threads" is the number of threads to be terminated. | |
| 3717 // "queue_set" is a set of work queues of other threads. | |
| 3718 // "collector" is the CMS collector associated with this task terminator. | |
| 3719 // "yield" indicates whether we need the gang as a whole to yield. | |
| 3720 CMSConcMarkingTerminator(int n_threads, TaskQueueSetSuper* queue_set, | |
| 3721 CMSCollector* collector, bool yield) : | |
| 3722 ParallelTaskTerminator(n_threads, queue_set), | |
| 3723 _collector(collector), | |
| 3724 _yield(yield) { } | |
| 3725 | |
| 3726 void set_task(CMSConcMarkingTask* task) { | |
| 3727 _task = task; | |
| 3728 } | |
| 3729 }; | |
| 3730 | |
| 3731 // MT Concurrent Marking Task | |
| 3732 class CMSConcMarkingTask: public YieldingFlexibleGangTask { | |
| 3733 CMSCollector* _collector; | |
| 3734 YieldingFlexibleWorkGang* _workers; // the whole gang | |
| 3735 int _n_workers; // requested/desired # workers | |
| 3736 bool _asynch; | |
| 3737 bool _result; | |
| 3738 CompactibleFreeListSpace* _cms_space; | |
| 3739 CompactibleFreeListSpace* _perm_space; | |
| 3740 HeapWord* _global_finger; | |
|
340
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3741 HeapWord* _restart_addr; |
| 0 | 3742 |
| 3743 // Exposed here for yielding support | |
| 3744 Mutex* const _bit_map_lock; | |
| 3745 | |
| 3746 // The per thread work queues, available here for stealing | |
| 3747 OopTaskQueueSet* _task_queues; | |
| 3748 CMSConcMarkingTerminator _term; | |
| 3749 | |
| 3750 public: | |
| 3751 CMSConcMarkingTask(CMSCollector* collector, | |
| 3752 CompactibleFreeListSpace* cms_space, | |
| 3753 CompactibleFreeListSpace* perm_space, | |
| 3754 bool asynch, int n_workers, | |
| 3755 YieldingFlexibleWorkGang* workers, | |
| 3756 OopTaskQueueSet* task_queues): | |
| 3757 YieldingFlexibleGangTask("Concurrent marking done multi-threaded"), | |
| 3758 _collector(collector), | |
| 3759 _cms_space(cms_space), | |
| 3760 _perm_space(perm_space), | |
| 3761 _asynch(asynch), _n_workers(n_workers), _result(true), | |
| 3762 _workers(workers), _task_queues(task_queues), | |
| 3763 _term(n_workers, task_queues, _collector, asynch), | |
| 3764 _bit_map_lock(collector->bitMapLock()) | |
| 3765 { | |
| 3766 assert(n_workers <= workers->total_workers(), | |
| 3767 "Else termination won't work correctly today"); // XXX FIX ME! | |
| 3768 _requested_size = n_workers; | |
| 3769 _term.set_task(this); | |
| 3770 assert(_cms_space->bottom() < _perm_space->bottom(), | |
| 3771 "Finger incorrectly initialized below"); | |
|
340
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3772 _restart_addr = _global_finger = _cms_space->bottom(); |
| 0 | 3773 } |
| 3774 | |
| 3775 | |
| 3776 OopTaskQueueSet* task_queues() { return _task_queues; } | |
| 3777 | |
| 3778 OopTaskQueue* work_queue(int i) { return task_queues()->queue(i); } | |
| 3779 | |
| 3780 HeapWord** global_finger_addr() { return &_global_finger; } | |
| 3781 | |
| 3782 CMSConcMarkingTerminator* terminator() { return &_term; } | |
| 3783 | |
| 3784 void work(int i); | |
| 3785 | |
| 3786 virtual void coordinator_yield(); // stuff done by coordinator | |
| 3787 bool result() { return _result; } | |
| 3788 | |
| 3789 void reset(HeapWord* ra) { | |
|
340
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3790 assert(_global_finger >= _cms_space->end(), "Postcondition of ::work(i)"); |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3791 assert(_global_finger >= _perm_space->end(), "Postcondition of ::work(i)"); |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3792 assert(ra < _perm_space->end(), "ra too large"); |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3793 _restart_addr = _global_finger = ra; |
| 0 | 3794 _term.reset_for_reuse(); |
| 3795 } | |
| 3796 | |
| 3797 static bool get_work_from_overflow_stack(CMSMarkStack* ovflw_stk, | |
| 3798 OopTaskQueue* work_q); | |
| 3799 | |
| 3800 private: | |
| 3801 void do_scan_and_mark(int i, CompactibleFreeListSpace* sp); | |
| 3802 void do_work_steal(int i); | |
| 3803 void bump_global_finger(HeapWord* f); | |
| 3804 }; | |
| 3805 | |
| 3806 void CMSConcMarkingTerminator::yield() { | |
| 3807 if (ConcurrentMarkSweepThread::should_yield() && | |
| 3808 !_collector->foregroundGCIsActive() && | |
| 3809 _yield) { | |
| 3810 _task->yield(); | |
| 3811 } else { | |
| 3812 ParallelTaskTerminator::yield(); | |
| 3813 } | |
| 3814 } | |
| 3815 | |
| 3816 //////////////////////////////////////////////////////////////// | |
| 3817 // Concurrent Marking Algorithm Sketch | |
| 3818 //////////////////////////////////////////////////////////////// | |
| 3819 // Until all tasks exhausted (both spaces): | |
| 3820 // -- claim next available chunk | |
| 3821 // -- bump global finger via CAS | |
| 3822 // -- find first object that starts in this chunk | |
| 3823 // and start scanning bitmap from that position | |
| 3824 // -- scan marked objects for oops | |
| 3825 // -- CAS-mark target, and if successful: | |
| 3826 // . if target oop is above global finger (volatile read) | |
| 3827 // nothing to do | |
| 3828 // . if target oop is in chunk and above local finger | |
| 3829 // then nothing to do | |
| 3830 // . else push on work-queue | |
| 3831 // -- Deal with possible overflow issues: | |
| 3832 // . local work-queue overflow causes stuff to be pushed on | |
| 3833 // global (common) overflow queue | |
| 3834 // . always first empty local work queue | |
| 3835 // . then get a batch of oops from global work queue if any | |
| 3836 // . then do work stealing | |
| 3837 // -- When all tasks claimed (both spaces) | |
| 3838 // and local work queue empty, | |
| 3839 // then in a loop do: | |
| 3840 // . check global overflow stack; steal a batch of oops and trace | |
| 3841 // . try to steal from other threads oif GOS is empty | |
| 3842 // . if neither is available, offer termination | |
| 3843 // -- Terminate and return result | |
| 3844 // | |
| 3845 void CMSConcMarkingTask::work(int i) { | |
| 3846 elapsedTimer _timer; | |
| 3847 ResourceMark rm; | |
| 3848 HandleMark hm; | |
| 3849 | |
| 3850 DEBUG_ONLY(_collector->verify_overflow_empty();) | |
| 3851 | |
| 3852 // Before we begin work, our work queue should be empty | |
| 3853 assert(work_queue(i)->size() == 0, "Expected to be empty"); | |
| 3854 // Scan the bitmap covering _cms_space, tracing through grey objects. | |
| 3855 _timer.start(); | |
| 3856 do_scan_and_mark(i, _cms_space); | |
| 3857 _timer.stop(); | |
| 3858 if (PrintCMSStatistics != 0) { | |
| 3859 gclog_or_tty->print_cr("Finished cms space scanning in %dth thread: %3.3f sec", | |
| 3860 i, _timer.seconds()); // XXX: need xxx/xxx type of notation, two timers | |
| 3861 } | |
| 3862 | |
| 3863 // ... do the same for the _perm_space | |
| 3864 _timer.reset(); | |
| 3865 _timer.start(); | |
| 3866 do_scan_and_mark(i, _perm_space); | |
| 3867 _timer.stop(); | |
| 3868 if (PrintCMSStatistics != 0) { | |
| 3869 gclog_or_tty->print_cr("Finished perm space scanning in %dth thread: %3.3f sec", | |
| 3870 i, _timer.seconds()); // XXX: need xxx/xxx type of notation, two timers | |
| 3871 } | |
| 3872 | |
| 3873 // ... do work stealing | |
| 3874 _timer.reset(); | |
| 3875 _timer.start(); | |
| 3876 do_work_steal(i); | |
| 3877 _timer.stop(); | |
| 3878 if (PrintCMSStatistics != 0) { | |
| 3879 gclog_or_tty->print_cr("Finished work stealing in %dth thread: %3.3f sec", | |
| 3880 i, _timer.seconds()); // XXX: need xxx/xxx type of notation, two timers | |
| 3881 } | |
| 3882 assert(_collector->_markStack.isEmpty(), "Should have been emptied"); | |
| 3883 assert(work_queue(i)->size() == 0, "Should have been emptied"); | |
| 3884 // Note that under the current task protocol, the | |
| 3885 // following assertion is true even of the spaces | |
| 3886 // expanded since the completion of the concurrent | |
| 3887 // marking. XXX This will likely change under a strict | |
| 3888 // ABORT semantics. | |
| 3889 assert(_global_finger > _cms_space->end() && | |
| 3890 _global_finger >= _perm_space->end(), | |
| 3891 "All tasks have been completed"); | |
| 3892 DEBUG_ONLY(_collector->verify_overflow_empty();) | |
| 3893 } | |
| 3894 | |
| 3895 void CMSConcMarkingTask::bump_global_finger(HeapWord* f) { | |
| 3896 HeapWord* read = _global_finger; | |
| 3897 HeapWord* cur = read; | |
| 3898 while (f > read) { | |
| 3899 cur = read; | |
| 3900 read = (HeapWord*) Atomic::cmpxchg_ptr(f, &_global_finger, cur); | |
| 3901 if (cur == read) { | |
| 3902 // our cas succeeded | |
| 3903 assert(_global_finger >= f, "protocol consistency"); | |
| 3904 break; | |
| 3905 } | |
| 3906 } | |
| 3907 } | |
| 3908 | |
| 3909 // This is really inefficient, and should be redone by | |
| 3910 // using (not yet available) block-read and -write interfaces to the | |
| 3911 // stack and the work_queue. XXX FIX ME !!! | |
| 3912 bool CMSConcMarkingTask::get_work_from_overflow_stack(CMSMarkStack* ovflw_stk, | |
| 3913 OopTaskQueue* work_q) { | |
| 3914 // Fast lock-free check | |
| 3915 if (ovflw_stk->length() == 0) { | |
| 3916 return false; | |
| 3917 } | |
| 3918 assert(work_q->size() == 0, "Shouldn't steal"); | |
| 3919 MutexLockerEx ml(ovflw_stk->par_lock(), | |
| 3920 Mutex::_no_safepoint_check_flag); | |
| 3921 // Grab up to 1/4 the size of the work queue | |
|
679
cea947c8a988
6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops
ysr
parents:
628
diff
changeset
|
3922 size_t num = MIN2((size_t)(work_q->max_elems() - work_q->size())/4, |
| 0 | 3923 (size_t)ParGCDesiredObjsFromOverflowList); |
| 3924 num = MIN2(num, ovflw_stk->length()); | |
| 3925 for (int i = (int) num; i > 0; i--) { | |
| 3926 oop cur = ovflw_stk->pop(); | |
| 3927 assert(cur != NULL, "Counted wrong?"); | |
| 3928 work_q->push(cur); | |
| 3929 } | |
| 3930 return num > 0; | |
| 3931 } | |
| 3932 | |
| 3933 void CMSConcMarkingTask::do_scan_and_mark(int i, CompactibleFreeListSpace* sp) { | |
| 3934 SequentialSubTasksDone* pst = sp->conc_par_seq_tasks(); | |
| 3935 int n_tasks = pst->n_tasks(); | |
| 3936 // We allow that there may be no tasks to do here because | |
| 3937 // we are restarting after a stack overflow. | |
|
340
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3938 assert(pst->valid() || n_tasks == 0, "Uninitialized use?"); |
| 0 | 3939 int nth_task = 0; |
| 3940 | |
|
340
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3941 HeapWord* aligned_start = sp->bottom(); |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3942 if (sp->used_region().contains(_restart_addr)) { |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3943 // Align down to a card boundary for the start of 0th task |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3944 // for this space. |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3945 aligned_start = |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3946 (HeapWord*)align_size_down((uintptr_t)_restart_addr, |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3947 CardTableModRefBS::card_size); |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3948 } |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3949 |
| 0 | 3950 size_t chunk_size = sp->marking_task_size(); |
| 3951 while (!pst->is_task_claimed(/* reference */ nth_task)) { | |
| 3952 // Having claimed the nth task in this space, | |
| 3953 // compute the chunk that it corresponds to: | |
|
340
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3954 MemRegion span = MemRegion(aligned_start + nth_task*chunk_size, |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3955 aligned_start + (nth_task+1)*chunk_size); |
| 0 | 3956 // Try and bump the global finger via a CAS; |
| 3957 // note that we need to do the global finger bump | |
| 3958 // _before_ taking the intersection below, because | |
| 3959 // the task corresponding to that region will be | |
| 3960 // deemed done even if the used_region() expands | |
| 3961 // because of allocation -- as it almost certainly will | |
| 3962 // during start-up while the threads yield in the | |
| 3963 // closure below. | |
| 3964 HeapWord* finger = span.end(); | |
| 3965 bump_global_finger(finger); // atomically | |
| 3966 // There are null tasks here corresponding to chunks | |
| 3967 // beyond the "top" address of the space. | |
| 3968 span = span.intersection(sp->used_region()); | |
| 3969 if (!span.is_empty()) { // Non-null task | |
|
340
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3970 HeapWord* prev_obj; |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3971 assert(!span.contains(_restart_addr) || nth_task == 0, |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3972 "Inconsistency"); |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3973 if (nth_task == 0) { |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3974 // For the 0th task, we'll not need to compute a block_start. |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3975 if (span.contains(_restart_addr)) { |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3976 // In the case of a restart because of stack overflow, |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3977 // we might additionally skip a chunk prefix. |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3978 prev_obj = _restart_addr; |
| 0 | 3979 } else { |
|
340
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3980 prev_obj = span.start(); |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3981 } |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3982 } else { |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3983 // We want to skip the first object because |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3984 // the protocol is to scan any object in its entirety |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3985 // that _starts_ in this span; a fortiori, any |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3986 // object starting in an earlier span is scanned |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3987 // as part of an earlier claimed task. |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3988 // Below we use the "careful" version of block_start |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3989 // so we do not try to navigate uninitialized objects. |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3990 prev_obj = sp->block_start_careful(span.start()); |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3991 // Below we use a variant of block_size that uses the |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3992 // Printezis bits to avoid waiting for allocated |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3993 // objects to become initialized/parsable. |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3994 while (prev_obj < span.start()) { |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3995 size_t sz = sp->block_size_no_stall(prev_obj, _collector); |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3996 if (sz > 0) { |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3997 prev_obj += sz; |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3998 } else { |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
3999 // In this case we may end up doing a bit of redundant |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
4000 // scanning, but that appears unavoidable, short of |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
4001 // locking the free list locks; see bug 6324141. |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
4002 break; |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
4003 } |
| 0 | 4004 } |
| 4005 } | |
| 4006 if (prev_obj < span.end()) { | |
| 4007 MemRegion my_span = MemRegion(prev_obj, span.end()); | |
| 4008 // Do the marking work within a non-empty span -- | |
| 4009 // the last argument to the constructor indicates whether the | |
| 4010 // iteration should be incremental with periodic yields. | |
| 4011 Par_MarkFromRootsClosure cl(this, _collector, my_span, | |
| 4012 &_collector->_markBitMap, | |
| 4013 work_queue(i), | |
| 4014 &_collector->_markStack, | |
| 4015 &_collector->_revisitStack, | |
| 4016 _asynch); | |
| 4017 _collector->_markBitMap.iterate(&cl, my_span.start(), my_span.end()); | |
| 4018 } // else nothing to do for this task | |
| 4019 } // else nothing to do for this task | |
| 4020 } | |
| 4021 // We'd be tempted to assert here that since there are no | |
| 4022 // more tasks left to claim in this space, the global_finger | |
| 4023 // must exceed space->top() and a fortiori space->end(). However, | |
| 4024 // that would not quite be correct because the bumping of | |
| 4025 // global_finger occurs strictly after the claiming of a task, | |
| 4026 // so by the time we reach here the global finger may not yet | |
| 4027 // have been bumped up by the thread that claimed the last | |
| 4028 // task. | |
| 4029 pst->all_tasks_completed(); | |
| 4030 } | |
| 4031 | |
| 935 | 4032 class Par_ConcMarkingClosure: public Par_KlassRememberingOopClosure { |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
4033 private: |
| 0 | 4034 MemRegion _span; |
| 4035 CMSBitMap* _bit_map; | |
| 4036 CMSMarkStack* _overflow_stack; | |
| 4037 OopTaskQueue* _work_queue; | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
4038 protected: |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
4039 DO_OOP_WORK_DEFN |
| 0 | 4040 public: |
| 4041 Par_ConcMarkingClosure(CMSCollector* collector, OopTaskQueue* work_queue, | |
| 935 | 4042 CMSBitMap* bit_map, CMSMarkStack* overflow_stack, |
| 4043 CMSMarkStack* revisit_stack): | |
| 4044 Par_KlassRememberingOopClosure(collector, NULL, revisit_stack), | |
| 0 | 4045 _span(_collector->_span), |
| 4046 _work_queue(work_queue), | |
| 4047 _bit_map(bit_map), | |
| 935 | 4048 _overflow_stack(overflow_stack) |
| 4049 { } | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
4050 virtual void do_oop(oop* p); |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
4051 virtual void do_oop(narrowOop* p); |
| 0 | 4052 void trim_queue(size_t max); |
| 4053 void handle_stack_overflow(HeapWord* lost); | |
| 4054 }; | |
| 4055 | |
|
340
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
4056 // Grey object scanning during work stealing phase -- |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
4057 // the salient assumption here is that any references |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
4058 // that are in these stolen objects being scanned must |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
4059 // already have been initialized (else they would not have |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
4060 // been published), so we do not need to check for |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
4061 // uninitialized objects before pushing here. |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
4062 void Par_ConcMarkingClosure::do_oop(oop obj) { |
|
340
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
4063 assert(obj->is_oop_or_null(true), "expected an oop or NULL"); |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
4064 HeapWord* addr = (HeapWord*)obj; |
| 0 | 4065 // Check if oop points into the CMS generation |
| 4066 // and is not marked | |
| 4067 if (_span.contains(addr) && !_bit_map->isMarked(addr)) { | |
| 4068 // a white object ... | |
| 4069 // If we manage to "claim" the object, by being the | |
| 4070 // first thread to mark it, then we push it on our | |
| 4071 // marking stack | |
| 4072 if (_bit_map->par_mark(addr)) { // ... now grey | |
| 4073 // push on work queue (grey set) | |
| 4074 bool simulate_overflow = false; | |
| 4075 NOT_PRODUCT( | |
| 4076 if (CMSMarkStackOverflowALot && | |
| 4077 _collector->simulate_overflow()) { | |
| 4078 // simulate a stack overflow | |
| 4079 simulate_overflow = true; | |
| 4080 } | |
| 4081 ) | |
| 4082 if (simulate_overflow || | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
4083 !(_work_queue->push(obj) || _overflow_stack->par_push(obj))) { |
| 0 | 4084 // stack overflow |
| 4085 if (PrintCMSStatistics != 0) { | |
| 4086 gclog_or_tty->print_cr("CMS marking stack overflow (benign) at " | |
| 4087 SIZE_FORMAT, _overflow_stack->capacity()); | |
| 4088 } | |
| 4089 // We cannot assert that the overflow stack is full because | |
| 4090 // it may have been emptied since. | |
| 4091 assert(simulate_overflow || | |
| 4092 _work_queue->size() == _work_queue->max_elems(), | |
| 4093 "Else push should have succeeded"); | |
| 4094 handle_stack_overflow(addr); | |
| 4095 } | |
| 4096 } // Else, some other thread got there first | |
| 4097 } | |
| 4098 } | |
| 4099 | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
4100 void Par_ConcMarkingClosure::do_oop(oop* p) { Par_ConcMarkingClosure::do_oop_work(p); } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
4101 void Par_ConcMarkingClosure::do_oop(narrowOop* p) { Par_ConcMarkingClosure::do_oop_work(p); } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
4102 |
| 0 | 4103 void Par_ConcMarkingClosure::trim_queue(size_t max) { |
| 4104 while (_work_queue->size() > max) { | |
| 4105 oop new_oop; | |
| 4106 if (_work_queue->pop_local(new_oop)) { | |
| 4107 assert(new_oop->is_oop(), "Should be an oop"); | |
| 4108 assert(_bit_map->isMarked((HeapWord*)new_oop), "Grey object"); | |
| 4109 assert(_span.contains((HeapWord*)new_oop), "Not in span"); | |
| 4110 assert(new_oop->is_parsable(), "Should be parsable"); | |
| 4111 new_oop->oop_iterate(this); // do_oop() above | |
| 4112 } | |
| 4113 } | |
| 4114 } | |
| 4115 | |
| 4116 // Upon stack overflow, we discard (part of) the stack, | |
| 4117 // remembering the least address amongst those discarded | |
| 4118 // in CMSCollector's _restart_address. | |
| 4119 void Par_ConcMarkingClosure::handle_stack_overflow(HeapWord* lost) { | |
| 4120 // We need to do this under a mutex to prevent other | |
|
340
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
4121 // workers from interfering with the work done below. |
| 0 | 4122 MutexLockerEx ml(_overflow_stack->par_lock(), |
| 4123 Mutex::_no_safepoint_check_flag); | |
| 4124 // Remember the least grey address discarded | |
| 4125 HeapWord* ra = (HeapWord*)_overflow_stack->least_value(lost); | |
| 4126 _collector->lower_restart_addr(ra); | |
| 4127 _overflow_stack->reset(); // discard stack contents | |
| 4128 _overflow_stack->expand(); // expand the stack if possible | |
| 4129 } | |
| 4130 | |
| 4131 | |
| 4132 void CMSConcMarkingTask::do_work_steal(int i) { | |
| 4133 OopTaskQueue* work_q = work_queue(i); | |
| 4134 oop obj_to_scan; | |
| 4135 CMSBitMap* bm = &(_collector->_markBitMap); | |
| 4136 CMSMarkStack* ovflw = &(_collector->_markStack); | |
| 935 | 4137 CMSMarkStack* revisit = &(_collector->_revisitStack); |
| 0 | 4138 int* seed = _collector->hash_seed(i); |
| 935 | 4139 Par_ConcMarkingClosure cl(_collector, work_q, bm, ovflw, revisit); |
| 0 | 4140 while (true) { |
| 4141 cl.trim_queue(0); | |
| 4142 assert(work_q->size() == 0, "Should have been emptied above"); | |
| 4143 if (get_work_from_overflow_stack(ovflw, work_q)) { | |
| 4144 // Can't assert below because the work obtained from the | |
| 4145 // overflow stack may already have been stolen from us. | |
| 4146 // assert(work_q->size() > 0, "Work from overflow stack"); | |
| 4147 continue; | |
| 4148 } else if (task_queues()->steal(i, seed, /* reference */ obj_to_scan)) { | |
| 4149 assert(obj_to_scan->is_oop(), "Should be an oop"); | |
| 4150 assert(bm->isMarked((HeapWord*)obj_to_scan), "Grey object"); | |
| 4151 obj_to_scan->oop_iterate(&cl); | |
| 4152 } else if (terminator()->offer_termination()) { | |
| 4153 assert(work_q->size() == 0, "Impossible!"); | |
| 4154 break; | |
| 4155 } | |
| 4156 } | |
| 4157 } | |
| 4158 | |
| 4159 // This is run by the CMS (coordinator) thread. | |
| 4160 void CMSConcMarkingTask::coordinator_yield() { | |
| 4161 assert(ConcurrentMarkSweepThread::cms_thread_has_cms_token(), | |
| 4162 "CMS thread should hold CMS token"); | |
| 935 | 4163 DEBUG_ONLY(RememberKlassesChecker mux(false);) |
| 0 | 4164 // First give up the locks, then yield, then re-lock |
| 4165 // We should probably use a constructor/destructor idiom to | |
| 4166 // do this unlock/lock or modify the MutexUnlocker class to | |
| 4167 // serve our purpose. XXX | |
| 4168 assert_lock_strong(_bit_map_lock); | |
| 4169 _bit_map_lock->unlock(); | |
| 4170 ConcurrentMarkSweepThread::desynchronize(true); | |
| 4171 ConcurrentMarkSweepThread::acknowledge_yield_request(); | |
| 4172 _collector->stopTimer(); | |
| 4173 if (PrintCMSStatistics != 0) { | |
| 4174 _collector->incrementYields(); | |
| 4175 } | |
| 4176 _collector->icms_wait(); | |
| 4177 | |
| 4178 // It is possible for whichever thread initiated the yield request | |
| 4179 // not to get a chance to wake up and take the bitmap lock between | |
| 4180 // this thread releasing it and reacquiring it. So, while the | |
| 4181 // should_yield() flag is on, let's sleep for a bit to give the | |
| 4182 // other thread a chance to wake up. The limit imposed on the number | |
| 4183 // of iterations is defensive, to avoid any unforseen circumstances | |
| 4184 // putting us into an infinite loop. Since it's always been this | |
| 4185 // (coordinator_yield()) method that was observed to cause the | |
| 4186 // problem, we are using a parameter (CMSCoordinatorYieldSleepCount) | |
| 4187 // which is by default non-zero. For the other seven methods that | |
| 4188 // also perform the yield operation, as are using a different | |
| 4189 // parameter (CMSYieldSleepCount) which is by default zero. This way we | |
| 4190 // can enable the sleeping for those methods too, if necessary. | |
| 4191 // See 6442774. | |
| 4192 // | |
| 4193 // We really need to reconsider the synchronization between the GC | |
| 4194 // thread and the yield-requesting threads in the future and we | |
| 4195 // should really use wait/notify, which is the recommended | |
| 4196 // way of doing this type of interaction. Additionally, we should | |
| 4197 // consolidate the eight methods that do the yield operation and they | |
| 4198 // are almost identical into one for better maintenability and | |
| 4199 // readability. See 6445193. | |
| 4200 // | |
| 4201 // Tony 2006.06.29 | |
| 4202 for (unsigned i = 0; i < CMSCoordinatorYieldSleepCount && | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
4203 ConcurrentMarkSweepThread::should_yield() && |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
4204 !CMSCollector::foregroundGCIsActive(); ++i) { |
| 0 | 4205 os::sleep(Thread::current(), 1, false); |
| 4206 ConcurrentMarkSweepThread::acknowledge_yield_request(); | |
| 4207 } | |
| 4208 | |
| 4209 ConcurrentMarkSweepThread::synchronize(true); | |
| 4210 _bit_map_lock->lock_without_safepoint_check(); | |
| 4211 _collector->startTimer(); | |
| 4212 } | |
| 4213 | |
| 4214 bool CMSCollector::do_marking_mt(bool asynch) { | |
| 1284 | 4215 assert(ConcGCThreads > 0 && conc_workers() != NULL, "precondition"); |
| 0 | 4216 // In the future this would be determined ergonomically, based |
| 4217 // on #cpu's, # active mutator threads (and load), and mutation rate. | |
| 1284 | 4218 int num_workers = ConcGCThreads; |
| 0 | 4219 |
| 4220 CompactibleFreeListSpace* cms_space = _cmsGen->cmsSpace(); | |
| 4221 CompactibleFreeListSpace* perm_space = _permGen->cmsSpace(); | |
| 4222 | |
| 4223 CMSConcMarkingTask tsk(this, cms_space, perm_space, | |
| 4224 asynch, num_workers /* number requested XXX */, | |
| 4225 conc_workers(), task_queues()); | |
| 4226 | |
| 4227 // Since the actual number of workers we get may be different | |
| 4228 // from the number we requested above, do we need to do anything different | |
| 4229 // below? In particular, may be we need to subclass the SequantialSubTasksDone | |
| 4230 // class?? XXX | |
| 4231 cms_space ->initialize_sequential_subtasks_for_marking(num_workers); | |
| 4232 perm_space->initialize_sequential_subtasks_for_marking(num_workers); | |
| 4233 | |
| 4234 // Refs discovery is already non-atomic. | |
| 4235 assert(!ref_processor()->discovery_is_atomic(), "Should be non-atomic"); | |
| 4236 // Mutate the Refs discovery so it is MT during the | |
| 4237 // multi-threaded marking phase. | |
| 4238 ReferenceProcessorMTMutator mt(ref_processor(), num_workers > 1); | |
|
1190
4788266644c1
6895236: CMS: cmsOopClosures.inline.hpp:43 assert(..., "Should remember klasses in this context")
jmasa
parents:
1145
diff
changeset
|
4239 DEBUG_ONLY(RememberKlassesChecker cmx(should_unload_classes());) |
| 0 | 4240 conc_workers()->start_task(&tsk); |
| 4241 while (tsk.yielded()) { | |
| 4242 tsk.coordinator_yield(); | |
| 4243 conc_workers()->continue_task(&tsk); | |
| 4244 } | |
| 4245 // If the task was aborted, _restart_addr will be non-NULL | |
| 4246 assert(tsk.completed() || _restart_addr != NULL, "Inconsistency"); | |
| 4247 while (_restart_addr != NULL) { | |
| 4248 // XXX For now we do not make use of ABORTED state and have not | |
| 4249 // yet implemented the right abort semantics (even in the original | |
| 4250 // single-threaded CMS case). That needs some more investigation | |
| 4251 // and is deferred for now; see CR# TBF. 07252005YSR. XXX | |
| 4252 assert(!CMSAbortSemantics || tsk.aborted(), "Inconsistency"); | |
| 4253 // If _restart_addr is non-NULL, a marking stack overflow | |
| 605 | 4254 // occurred; we need to do a fresh marking iteration from the |
| 0 | 4255 // indicated restart address. |
| 4256 if (_foregroundGCIsActive && asynch) { | |
| 4257 // We may be running into repeated stack overflows, having | |
| 4258 // reached the limit of the stack size, while making very | |
| 4259 // slow forward progress. It may be best to bail out and | |
| 4260 // let the foreground collector do its job. | |
| 4261 // Clear _restart_addr, so that foreground GC | |
| 4262 // works from scratch. This avoids the headache of | |
| 4263 // a "rescan" which would otherwise be needed because | |
| 4264 // of the dirty mod union table & card table. | |
| 4265 _restart_addr = NULL; | |
| 4266 return false; | |
| 4267 } | |
| 4268 // Adjust the task to restart from _restart_addr | |
| 4269 tsk.reset(_restart_addr); | |
| 4270 cms_space ->initialize_sequential_subtasks_for_marking(num_workers, | |
| 4271 _restart_addr); | |
| 4272 perm_space->initialize_sequential_subtasks_for_marking(num_workers, | |
| 4273 _restart_addr); | |
| 4274 _restart_addr = NULL; | |
| 4275 // Get the workers going again | |
| 4276 conc_workers()->start_task(&tsk); | |
| 4277 while (tsk.yielded()) { | |
| 4278 tsk.coordinator_yield(); | |
| 4279 conc_workers()->continue_task(&tsk); | |
| 4280 } | |
| 4281 } | |
| 4282 assert(tsk.completed(), "Inconsistency"); | |
| 4283 assert(tsk.result() == true, "Inconsistency"); | |
| 4284 return true; | |
| 4285 } | |
| 4286 | |
| 4287 bool CMSCollector::do_marking_st(bool asynch) { | |
| 4288 ResourceMark rm; | |
| 4289 HandleMark hm; | |
| 4290 | |
| 4291 MarkFromRootsClosure markFromRootsClosure(this, _span, &_markBitMap, | |
| 4292 &_markStack, &_revisitStack, CMSYield && asynch); | |
| 4293 // the last argument to iterate indicates whether the iteration | |
| 4294 // should be incremental with periodic yields. | |
| 4295 _markBitMap.iterate(&markFromRootsClosure); | |
| 4296 // If _restart_addr is non-NULL, a marking stack overflow | |
| 605 | 4297 // occurred; we need to do a fresh iteration from the |
| 0 | 4298 // indicated restart address. |
| 4299 while (_restart_addr != NULL) { | |
| 4300 if (_foregroundGCIsActive && asynch) { | |
| 4301 // We may be running into repeated stack overflows, having | |
| 4302 // reached the limit of the stack size, while making very | |
| 4303 // slow forward progress. It may be best to bail out and | |
| 4304 // let the foreground collector do its job. | |
| 4305 // Clear _restart_addr, so that foreground GC | |
| 4306 // works from scratch. This avoids the headache of | |
| 4307 // a "rescan" which would otherwise be needed because | |
| 4308 // of the dirty mod union table & card table. | |
| 4309 _restart_addr = NULL; | |
| 4310 return false; // indicating failure to complete marking | |
| 4311 } | |
| 4312 // Deal with stack overflow: | |
| 4313 // we restart marking from _restart_addr | |
| 4314 HeapWord* ra = _restart_addr; | |
| 4315 markFromRootsClosure.reset(ra); | |
| 4316 _restart_addr = NULL; | |
| 4317 _markBitMap.iterate(&markFromRootsClosure, ra, _span.end()); | |
| 4318 } | |
| 4319 return true; | |
| 4320 } | |
| 4321 | |
| 4322 void CMSCollector::preclean() { | |
| 4323 check_correct_thread_executing(); | |
| 4324 assert(Thread::current()->is_ConcurrentGC_thread(), "Wrong thread"); | |
| 4325 verify_work_stacks_empty(); | |
| 4326 verify_overflow_empty(); | |
| 4327 _abort_preclean = false; | |
| 4328 if (CMSPrecleaningEnabled) { | |
| 4329 _eden_chunk_index = 0; | |
| 4330 size_t used = get_eden_used(); | |
| 4331 size_t capacity = get_eden_capacity(); | |
| 4332 // Don't start sampling unless we will get sufficiently | |
| 4333 // many samples. | |
| 4334 if (used < (capacity/(CMSScheduleRemarkSamplingRatio * 100) | |
| 4335 * CMSScheduleRemarkEdenPenetration)) { | |
| 4336 _start_sampling = true; | |
| 4337 } else { | |
| 4338 _start_sampling = false; | |
| 4339 } | |
| 4340 TraceCPUTime tcpu(PrintGCDetails, true, gclog_or_tty); | |
| 4341 CMSPhaseAccounting pa(this, "preclean", !PrintGCDetails); | |
| 4342 preclean_work(CMSPrecleanRefLists1, CMSPrecleanSurvivors1); | |
| 4343 } | |
| 4344 CMSTokenSync x(true); // is cms thread | |
| 4345 if (CMSPrecleaningEnabled) { | |
| 4346 sample_eden(); | |
| 4347 _collectorState = AbortablePreclean; | |
| 4348 } else { | |
| 4349 _collectorState = FinalMarking; | |
| 4350 } | |
| 4351 verify_work_stacks_empty(); | |
| 4352 verify_overflow_empty(); | |
| 4353 } | |
| 4354 | |
| 4355 // Try and schedule the remark such that young gen | |
| 4356 // occupancy is CMSScheduleRemarkEdenPenetration %. | |
| 4357 void CMSCollector::abortable_preclean() { | |
| 4358 check_correct_thread_executing(); | |
| 4359 assert(CMSPrecleaningEnabled, "Inconsistent control state"); | |
| 4360 assert(_collectorState == AbortablePreclean, "Inconsistent control state"); | |
| 4361 | |
| 4362 // If Eden's current occupancy is below this threshold, | |
| 4363 // immediately schedule the remark; else preclean | |
| 4364 // past the next scavenge in an effort to | |
| 4365 // schedule the pause as described avove. By choosing | |
| 4366 // CMSScheduleRemarkEdenSizeThreshold >= max eden size | |
| 4367 // we will never do an actual abortable preclean cycle. | |
| 4368 if (get_eden_used() > CMSScheduleRemarkEdenSizeThreshold) { | |
| 4369 TraceCPUTime tcpu(PrintGCDetails, true, gclog_or_tty); | |
| 4370 CMSPhaseAccounting pa(this, "abortable-preclean", !PrintGCDetails); | |
| 4371 // We need more smarts in the abortable preclean | |
| 4372 // loop below to deal with cases where allocation | |
| 4373 // in young gen is very very slow, and our precleaning | |
| 4374 // is running a losing race against a horde of | |
| 4375 // mutators intent on flooding us with CMS updates | |
| 4376 // (dirty cards). | |
| 4377 // One, admittedly dumb, strategy is to give up | |
| 4378 // after a certain number of abortable precleaning loops | |
| 4379 // or after a certain maximum time. We want to make | |
| 4380 // this smarter in the next iteration. | |
| 4381 // XXX FIX ME!!! YSR | |
| 4382 size_t loops = 0, workdone = 0, cumworkdone = 0, waited = 0; | |
| 4383 while (!(should_abort_preclean() || | |
| 4384 ConcurrentMarkSweepThread::should_terminate())) { | |
| 4385 workdone = preclean_work(CMSPrecleanRefLists2, CMSPrecleanSurvivors2); | |
| 4386 cumworkdone += workdone; | |
| 4387 loops++; | |
| 4388 // Voluntarily terminate abortable preclean phase if we have | |
| 4389 // been at it for too long. | |
| 4390 if ((CMSMaxAbortablePrecleanLoops != 0) && | |
| 4391 loops >= CMSMaxAbortablePrecleanLoops) { | |
| 4392 if (PrintGCDetails) { | |
| 4393 gclog_or_tty->print(" CMS: abort preclean due to loops "); | |
| 4394 } | |
| 4395 break; | |
| 4396 } | |
| 4397 if (pa.wallclock_millis() > CMSMaxAbortablePrecleanTime) { | |
| 4398 if (PrintGCDetails) { | |
| 4399 gclog_or_tty->print(" CMS: abort preclean due to time "); | |
| 4400 } | |
| 4401 break; | |
| 4402 } | |
| 4403 // If we are doing little work each iteration, we should | |
| 4404 // take a short break. | |
| 4405 if (workdone < CMSAbortablePrecleanMinWorkPerIteration) { | |
| 4406 // Sleep for some time, waiting for work to accumulate | |
| 4407 stopTimer(); | |
| 4408 cmsThread()->wait_on_cms_lock(CMSAbortablePrecleanWaitMillis); | |
| 4409 startTimer(); | |
| 4410 waited++; | |
| 4411 } | |
| 4412 } | |
| 4413 if (PrintCMSStatistics > 0) { | |
| 4414 gclog_or_tty->print(" [%d iterations, %d waits, %d cards)] ", | |
| 4415 loops, waited, cumworkdone); | |
| 4416 } | |
| 4417 } | |
| 4418 CMSTokenSync x(true); // is cms thread | |
| 4419 if (_collectorState != Idling) { | |
| 4420 assert(_collectorState == AbortablePreclean, | |
| 4421 "Spontaneous state transition?"); | |
| 4422 _collectorState = FinalMarking; | |
| 4423 } // Else, a foreground collection completed this CMS cycle. | |
| 4424 return; | |
| 4425 } | |
| 4426 | |
| 4427 // Respond to an Eden sampling opportunity | |
| 4428 void CMSCollector::sample_eden() { | |
| 4429 // Make sure a young gc cannot sneak in between our | |
| 4430 // reading and recording of a sample. | |
| 4431 assert(Thread::current()->is_ConcurrentGC_thread(), | |
| 4432 "Only the cms thread may collect Eden samples"); | |
| 4433 assert(ConcurrentMarkSweepThread::cms_thread_has_cms_token(), | |
| 4434 "Should collect samples while holding CMS token"); | |
| 4435 if (!_start_sampling) { | |
| 4436 return; | |
| 4437 } | |
| 4438 if (_eden_chunk_array) { | |
| 4439 if (_eden_chunk_index < _eden_chunk_capacity) { | |
| 4440 _eden_chunk_array[_eden_chunk_index] = *_top_addr; // take sample | |
| 4441 assert(_eden_chunk_array[_eden_chunk_index] <= *_end_addr, | |
| 4442 "Unexpected state of Eden"); | |
| 4443 // We'd like to check that what we just sampled is an oop-start address; | |
| 4444 // however, we cannot do that here since the object may not yet have been | |
| 4445 // initialized. So we'll instead do the check when we _use_ this sample | |
| 4446 // later. | |
| 4447 if (_eden_chunk_index == 0 || | |
| 4448 (pointer_delta(_eden_chunk_array[_eden_chunk_index], | |
| 4449 _eden_chunk_array[_eden_chunk_index-1]) | |
| 4450 >= CMSSamplingGrain)) { | |
| 4451 _eden_chunk_index++; // commit sample | |
| 4452 } | |
| 4453 } | |
| 4454 } | |
| 4455 if ((_collectorState == AbortablePreclean) && !_abort_preclean) { | |
| 4456 size_t used = get_eden_used(); | |
| 4457 size_t capacity = get_eden_capacity(); | |
| 4458 assert(used <= capacity, "Unexpected state of Eden"); | |
| 4459 if (used > (capacity/100 * CMSScheduleRemarkEdenPenetration)) { | |
| 4460 _abort_preclean = true; | |
| 4461 } | |
| 4462 } | |
| 4463 } | |
| 4464 | |
| 4465 | |
| 4466 size_t CMSCollector::preclean_work(bool clean_refs, bool clean_survivor) { | |
| 4467 assert(_collectorState == Precleaning || | |
| 4468 _collectorState == AbortablePreclean, "incorrect state"); | |
| 4469 ResourceMark rm; | |
| 4470 HandleMark hm; | |
| 4471 // Do one pass of scrubbing the discovered reference lists | |
| 4472 // to remove any reference objects with strongly-reachable | |
| 4473 // referents. | |
| 4474 if (clean_refs) { | |
| 4475 ReferenceProcessor* rp = ref_processor(); | |
| 4476 CMSPrecleanRefsYieldClosure yield_cl(this); | |
| 4477 assert(rp->span().equals(_span), "Spans should be equal"); | |
| 4478 CMSKeepAliveClosure keep_alive(this, _span, &_markBitMap, | |
| 935 | 4479 &_markStack, &_revisitStack, |
| 4480 true /* preclean */); | |
| 0 | 4481 CMSDrainMarkingStackClosure complete_trace(this, |
|
452
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
4482 _span, &_markBitMap, &_markStack, |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
4483 &keep_alive, true /* preclean */); |
| 0 | 4484 |
| 4485 // We don't want this step to interfere with a young | |
| 4486 // collection because we don't want to take CPU | |
| 4487 // or memory bandwidth away from the young GC threads | |
| 4488 // (which may be as many as there are CPUs). | |
| 4489 // Note that we don't need to protect ourselves from | |
| 4490 // interference with mutators because they can't | |
| 4491 // manipulate the discovered reference lists nor affect | |
| 4492 // the computed reachability of the referents, the | |
| 4493 // only properties manipulated by the precleaning | |
| 4494 // of these reference lists. | |
| 4495 stopTimer(); | |
| 4496 CMSTokenSyncWithLocks x(true /* is cms thread */, | |
| 4497 bitMapLock()); | |
| 4498 startTimer(); | |
| 4499 sample_eden(); | |
| 935 | 4500 |
| 0 | 4501 // The following will yield to allow foreground |
| 4502 // collection to proceed promptly. XXX YSR: | |
| 4503 // The code in this method may need further | |
| 4504 // tweaking for better performance and some restructuring | |
| 4505 // for cleaner interfaces. | |
| 4506 rp->preclean_discovered_references( | |
| 4507 rp->is_alive_non_header(), &keep_alive, &complete_trace, | |
|
1190
4788266644c1
6895236: CMS: cmsOopClosures.inline.hpp:43 assert(..., "Should remember klasses in this context")
jmasa
parents:
1145
diff
changeset
|
4508 &yield_cl, should_unload_classes()); |
| 0 | 4509 } |
| 4510 | |
| 4511 if (clean_survivor) { // preclean the active survivor space(s) | |
| 4512 assert(_young_gen->kind() == Generation::DefNew || | |
| 4513 _young_gen->kind() == Generation::ParNew || | |
| 4514 _young_gen->kind() == Generation::ASParNew, | |
| 4515 "incorrect type for cast"); | |
| 4516 DefNewGeneration* dng = (DefNewGeneration*)_young_gen; | |
| 4517 PushAndMarkClosure pam_cl(this, _span, ref_processor(), | |
| 4518 &_markBitMap, &_modUnionTable, | |
| 4519 &_markStack, &_revisitStack, | |
| 4520 true /* precleaning phase */); | |
| 4521 stopTimer(); | |
| 4522 CMSTokenSyncWithLocks ts(true /* is cms thread */, | |
| 4523 bitMapLock()); | |
| 4524 startTimer(); | |
| 4525 unsigned int before_count = | |
| 4526 GenCollectedHeap::heap()->total_collections(); | |
| 4527 SurvivorSpacePrecleanClosure | |
| 4528 sss_cl(this, _span, &_markBitMap, &_markStack, | |
| 4529 &pam_cl, before_count, CMSYield); | |
|
1190
4788266644c1
6895236: CMS: cmsOopClosures.inline.hpp:43 assert(..., "Should remember klasses in this context")
jmasa
parents:
1145
diff
changeset
|
4530 DEBUG_ONLY(RememberKlassesChecker mx(should_unload_classes());) |
| 0 | 4531 dng->from()->object_iterate_careful(&sss_cl); |
| 4532 dng->to()->object_iterate_careful(&sss_cl); | |
| 4533 } | |
| 4534 MarkRefsIntoAndScanClosure | |
| 4535 mrias_cl(_span, ref_processor(), &_markBitMap, &_modUnionTable, | |
| 4536 &_markStack, &_revisitStack, this, CMSYield, | |
| 4537 true /* precleaning phase */); | |
| 4538 // CAUTION: The following closure has persistent state that may need to | |
| 4539 // be reset upon a decrease in the sequence of addresses it | |
| 4540 // processes. | |
| 4541 ScanMarkedObjectsAgainCarefullyClosure | |
| 4542 smoac_cl(this, _span, | |
| 4543 &_markBitMap, &_markStack, &_revisitStack, &mrias_cl, CMSYield); | |
| 4544 | |
| 4545 // Preclean dirty cards in ModUnionTable and CardTable using | |
| 4546 // appropriate convergence criterion; | |
| 4547 // repeat CMSPrecleanIter times unless we find that | |
| 4548 // we are losing. | |
| 4549 assert(CMSPrecleanIter < 10, "CMSPrecleanIter is too large"); | |
| 4550 assert(CMSPrecleanNumerator < CMSPrecleanDenominator, | |
| 4551 "Bad convergence multiplier"); | |
| 4552 assert(CMSPrecleanThreshold >= 100, | |
| 4553 "Unreasonably low CMSPrecleanThreshold"); | |
| 4554 | |
| 4555 size_t numIter, cumNumCards, lastNumCards, curNumCards; | |
| 4556 for (numIter = 0, cumNumCards = lastNumCards = curNumCards = 0; | |
| 4557 numIter < CMSPrecleanIter; | |
| 4558 numIter++, lastNumCards = curNumCards, cumNumCards += curNumCards) { | |
| 4559 curNumCards = preclean_mod_union_table(_cmsGen, &smoac_cl); | |
| 4560 if (CMSPermGenPrecleaningEnabled) { | |
| 4561 curNumCards += preclean_mod_union_table(_permGen, &smoac_cl); | |
| 4562 } | |
| 4563 if (Verbose && PrintGCDetails) { | |
| 4564 gclog_or_tty->print(" (modUnionTable: %d cards)", curNumCards); | |
| 4565 } | |
| 4566 // Either there are very few dirty cards, so re-mark | |
| 4567 // pause will be small anyway, or our pre-cleaning isn't | |
| 4568 // that much faster than the rate at which cards are being | |
| 4569 // dirtied, so we might as well stop and re-mark since | |
| 4570 // precleaning won't improve our re-mark time by much. | |
| 4571 if (curNumCards <= CMSPrecleanThreshold || | |
| 4572 (numIter > 0 && | |
| 4573 (curNumCards * CMSPrecleanDenominator > | |
| 4574 lastNumCards * CMSPrecleanNumerator))) { | |
| 4575 numIter++; | |
| 4576 cumNumCards += curNumCards; | |
| 4577 break; | |
| 4578 } | |
| 4579 } | |
| 4580 curNumCards = preclean_card_table(_cmsGen, &smoac_cl); | |
| 4581 if (CMSPermGenPrecleaningEnabled) { | |
| 4582 curNumCards += preclean_card_table(_permGen, &smoac_cl); | |
| 4583 } | |
| 4584 cumNumCards += curNumCards; | |
| 4585 if (PrintGCDetails && PrintCMSStatistics != 0) { | |
| 4586 gclog_or_tty->print_cr(" (cardTable: %d cards, re-scanned %d cards, %d iterations)", | |
| 4587 curNumCards, cumNumCards, numIter); | |
| 4588 } | |
| 4589 return cumNumCards; // as a measure of useful work done | |
| 4590 } | |
| 4591 | |
| 4592 // PRECLEANING NOTES: | |
| 4593 // Precleaning involves: | |
| 4594 // . reading the bits of the modUnionTable and clearing the set bits. | |
| 4595 // . For the cards corresponding to the set bits, we scan the | |
| 4596 // objects on those cards. This means we need the free_list_lock | |
| 4597 // so that we can safely iterate over the CMS space when scanning | |
| 4598 // for oops. | |
| 4599 // . When we scan the objects, we'll be both reading and setting | |
| 4600 // marks in the marking bit map, so we'll need the marking bit map. | |
| 4601 // . For protecting _collector_state transitions, we take the CGC_lock. | |
| 4602 // Note that any races in the reading of of card table entries by the | |
| 4603 // CMS thread on the one hand and the clearing of those entries by the | |
| 4604 // VM thread or the setting of those entries by the mutator threads on the | |
| 4605 // other are quite benign. However, for efficiency it makes sense to keep | |
| 4606 // the VM thread from racing with the CMS thread while the latter is | |
| 4607 // dirty card info to the modUnionTable. We therefore also use the | |
| 4608 // CGC_lock to protect the reading of the card table and the mod union | |
| 4609 // table by the CM thread. | |
| 4610 // . We run concurrently with mutator updates, so scanning | |
| 4611 // needs to be done carefully -- we should not try to scan | |
| 4612 // potentially uninitialized objects. | |
| 4613 // | |
| 4614 // Locking strategy: While holding the CGC_lock, we scan over and | |
| 4615 // reset a maximal dirty range of the mod union / card tables, then lock | |
| 4616 // the free_list_lock and bitmap lock to do a full marking, then | |
| 4617 // release these locks; and repeat the cycle. This allows for a | |
| 4618 // certain amount of fairness in the sharing of these locks between | |
| 4619 // the CMS collector on the one hand, and the VM thread and the | |
| 4620 // mutators on the other. | |
| 4621 | |
| 4622 // NOTE: preclean_mod_union_table() and preclean_card_table() | |
| 4623 // further below are largely identical; if you need to modify | |
| 4624 // one of these methods, please check the other method too. | |
| 4625 | |
| 4626 size_t CMSCollector::preclean_mod_union_table( | |
| 4627 ConcurrentMarkSweepGeneration* gen, | |
| 4628 ScanMarkedObjectsAgainCarefullyClosure* cl) { | |
| 4629 verify_work_stacks_empty(); | |
| 4630 verify_overflow_empty(); | |
| 4631 | |
| 935 | 4632 // Turn off checking for this method but turn it back on |
| 4633 // selectively. There are yield points in this method | |
| 4634 // but it is difficult to turn the checking off just around | |
| 4635 // the yield points. It is simpler to selectively turn | |
| 4636 // it on. | |
| 4637 DEBUG_ONLY(RememberKlassesChecker mux(false);) | |
| 4638 | |
| 0 | 4639 // strategy: starting with the first card, accumulate contiguous |
| 4640 // ranges of dirty cards; clear these cards, then scan the region | |
| 4641 // covered by these cards. | |
| 4642 | |
| 4643 // Since all of the MUT is committed ahead, we can just use | |
| 4644 // that, in case the generations expand while we are precleaning. | |
| 4645 // It might also be fine to just use the committed part of the | |
| 4646 // generation, but we might potentially miss cards when the | |
| 4647 // generation is rapidly expanding while we are in the midst | |
| 4648 // of precleaning. | |
| 4649 HeapWord* startAddr = gen->reserved().start(); | |
| 4650 HeapWord* endAddr = gen->reserved().end(); | |
| 4651 | |
| 4652 cl->setFreelistLock(gen->freelistLock()); // needed for yielding | |
| 4653 | |
| 4654 size_t numDirtyCards, cumNumDirtyCards; | |
| 4655 HeapWord *nextAddr, *lastAddr; | |
| 4656 for (cumNumDirtyCards = numDirtyCards = 0, | |
| 4657 nextAddr = lastAddr = startAddr; | |
| 4658 nextAddr < endAddr; | |
| 4659 nextAddr = lastAddr, cumNumDirtyCards += numDirtyCards) { | |
| 4660 | |
| 4661 ResourceMark rm; | |
| 4662 HandleMark hm; | |
| 4663 | |
| 4664 MemRegion dirtyRegion; | |
| 4665 { | |
| 4666 stopTimer(); | |
| 935 | 4667 // Potential yield point |
| 0 | 4668 CMSTokenSync ts(true); |
| 4669 startTimer(); | |
| 4670 sample_eden(); | |
| 4671 // Get dirty region starting at nextOffset (inclusive), | |
| 4672 // simultaneously clearing it. | |
| 4673 dirtyRegion = | |
| 4674 _modUnionTable.getAndClearMarkedRegion(nextAddr, endAddr); | |
| 4675 assert(dirtyRegion.start() >= nextAddr, | |
| 4676 "returned region inconsistent?"); | |
| 4677 } | |
| 4678 // Remember where the next search should begin. | |
| 4679 // The returned region (if non-empty) is a right open interval, | |
| 4680 // so lastOffset is obtained from the right end of that | |
| 4681 // interval. | |
| 4682 lastAddr = dirtyRegion.end(); | |
| 4683 // Should do something more transparent and less hacky XXX | |
| 4684 numDirtyCards = | |
| 4685 _modUnionTable.heapWordDiffToOffsetDiff(dirtyRegion.word_size()); | |
| 4686 | |
| 4687 // We'll scan the cards in the dirty region (with periodic | |
| 4688 // yields for foreground GC as needed). | |
| 4689 if (!dirtyRegion.is_empty()) { | |
| 4690 assert(numDirtyCards > 0, "consistency check"); | |
| 4691 HeapWord* stop_point = NULL; | |
|
453
c96030fff130
6684579: SoftReference processing can be made more efficient
ysr
parents:
452
diff
changeset
|
4692 stopTimer(); |
| 935 | 4693 // Potential yield point |
|
453
c96030fff130
6684579: SoftReference processing can be made more efficient
ysr
parents:
452
diff
changeset
|
4694 CMSTokenSyncWithLocks ts(true, gen->freelistLock(), |
|
c96030fff130
6684579: SoftReference processing can be made more efficient
ysr
parents:
452
diff
changeset
|
4695 bitMapLock()); |
|
c96030fff130
6684579: SoftReference processing can be made more efficient
ysr
parents:
452
diff
changeset
|
4696 startTimer(); |
| 0 | 4697 { |
| 4698 verify_work_stacks_empty(); | |
| 4699 verify_overflow_empty(); | |
| 4700 sample_eden(); | |
|
1190
4788266644c1
6895236: CMS: cmsOopClosures.inline.hpp:43 assert(..., "Should remember klasses in this context")
jmasa
parents:
1145
diff
changeset
|
4701 DEBUG_ONLY(RememberKlassesChecker mx(should_unload_classes());) |
| 0 | 4702 stop_point = |
| 4703 gen->cmsSpace()->object_iterate_careful_m(dirtyRegion, cl); | |
| 4704 } | |
| 4705 if (stop_point != NULL) { | |
| 4706 // The careful iteration stopped early either because it found an | |
| 4707 // uninitialized object, or because we were in the midst of an | |
| 4708 // "abortable preclean", which should now be aborted. Redirty | |
| 4709 // the bits corresponding to the partially-scanned or unscanned | |
| 4710 // cards. We'll either restart at the next block boundary or | |
| 4711 // abort the preclean. | |
| 4712 assert((CMSPermGenPrecleaningEnabled && (gen == _permGen)) || | |
| 4713 (_collectorState == AbortablePreclean && should_abort_preclean()), | |
| 4714 "Unparsable objects should only be in perm gen."); | |
| 4715 _modUnionTable.mark_range(MemRegion(stop_point, dirtyRegion.end())); | |
| 4716 if (should_abort_preclean()) { | |
| 4717 break; // out of preclean loop | |
| 4718 } else { | |
| 4719 // Compute the next address at which preclean should pick up; | |
| 4720 // might need bitMapLock in order to read P-bits. | |
| 4721 lastAddr = next_card_start_after_block(stop_point); | |
| 4722 } | |
| 4723 } | |
| 4724 } else { | |
| 4725 assert(lastAddr == endAddr, "consistency check"); | |
| 4726 assert(numDirtyCards == 0, "consistency check"); | |
| 4727 break; | |
| 4728 } | |
| 4729 } | |
| 4730 verify_work_stacks_empty(); | |
| 4731 verify_overflow_empty(); | |
| 4732 return cumNumDirtyCards; | |
| 4733 } | |
| 4734 | |
| 4735 // NOTE: preclean_mod_union_table() above and preclean_card_table() | |
| 4736 // below are largely identical; if you need to modify | |
| 4737 // one of these methods, please check the other method too. | |
| 4738 | |
| 4739 size_t CMSCollector::preclean_card_table(ConcurrentMarkSweepGeneration* gen, | |
| 4740 ScanMarkedObjectsAgainCarefullyClosure* cl) { | |
| 4741 // strategy: it's similar to precleamModUnionTable above, in that | |
| 4742 // we accumulate contiguous ranges of dirty cards, mark these cards | |
| 4743 // precleaned, then scan the region covered by these cards. | |
| 4744 HeapWord* endAddr = (HeapWord*)(gen->_virtual_space.high()); | |
| 4745 HeapWord* startAddr = (HeapWord*)(gen->_virtual_space.low()); | |
| 4746 | |
| 4747 cl->setFreelistLock(gen->freelistLock()); // needed for yielding | |
| 4748 | |
| 4749 size_t numDirtyCards, cumNumDirtyCards; | |
| 4750 HeapWord *lastAddr, *nextAddr; | |
| 4751 | |
| 4752 for (cumNumDirtyCards = numDirtyCards = 0, | |
| 4753 nextAddr = lastAddr = startAddr; | |
| 4754 nextAddr < endAddr; | |
| 4755 nextAddr = lastAddr, cumNumDirtyCards += numDirtyCards) { | |
| 4756 | |
| 4757 ResourceMark rm; | |
| 4758 HandleMark hm; | |
| 4759 | |
| 4760 MemRegion dirtyRegion; | |
| 4761 { | |
| 4762 // See comments in "Precleaning notes" above on why we | |
| 4763 // do this locking. XXX Could the locking overheads be | |
| 4764 // too high when dirty cards are sparse? [I don't think so.] | |
| 4765 stopTimer(); | |
| 4766 CMSTokenSync x(true); // is cms thread | |
| 4767 startTimer(); | |
| 4768 sample_eden(); | |
| 4769 // Get and clear dirty region from card table | |
|
342
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
4770 dirtyRegion = _ct->ct_bs()->dirty_card_range_after_reset( |
|
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
4771 MemRegion(nextAddr, endAddr), |
|
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
4772 true, |
|
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
4773 CardTableModRefBS::precleaned_card_val()); |
|
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
4774 |
| 0 | 4775 assert(dirtyRegion.start() >= nextAddr, |
| 4776 "returned region inconsistent?"); | |
| 4777 } | |
| 4778 lastAddr = dirtyRegion.end(); | |
| 4779 numDirtyCards = | |
| 4780 dirtyRegion.word_size()/CardTableModRefBS::card_size_in_words; | |
| 4781 | |
| 4782 if (!dirtyRegion.is_empty()) { | |
| 4783 stopTimer(); | |
| 4784 CMSTokenSyncWithLocks ts(true, gen->freelistLock(), bitMapLock()); | |
| 4785 startTimer(); | |
| 4786 sample_eden(); | |
| 4787 verify_work_stacks_empty(); | |
| 4788 verify_overflow_empty(); | |
|
1190
4788266644c1
6895236: CMS: cmsOopClosures.inline.hpp:43 assert(..., "Should remember klasses in this context")
jmasa
parents:
1145
diff
changeset
|
4789 DEBUG_ONLY(RememberKlassesChecker mx(should_unload_classes());) |
| 0 | 4790 HeapWord* stop_point = |
| 4791 gen->cmsSpace()->object_iterate_careful_m(dirtyRegion, cl); | |
| 4792 if (stop_point != NULL) { | |
| 4793 // The careful iteration stopped early because it found an | |
| 4794 // uninitialized object. Redirty the bits corresponding to the | |
| 4795 // partially-scanned or unscanned cards, and start again at the | |
| 4796 // next block boundary. | |
| 4797 assert(CMSPermGenPrecleaningEnabled || | |
| 4798 (_collectorState == AbortablePreclean && should_abort_preclean()), | |
| 4799 "Unparsable objects should only be in perm gen."); | |
| 4800 _ct->ct_bs()->invalidate(MemRegion(stop_point, dirtyRegion.end())); | |
| 4801 if (should_abort_preclean()) { | |
| 4802 break; // out of preclean loop | |
| 4803 } else { | |
| 4804 // Compute the next address at which preclean should pick up. | |
| 4805 lastAddr = next_card_start_after_block(stop_point); | |
| 4806 } | |
| 4807 } | |
| 4808 } else { | |
| 4809 break; | |
| 4810 } | |
| 4811 } | |
| 4812 verify_work_stacks_empty(); | |
| 4813 verify_overflow_empty(); | |
| 4814 return cumNumDirtyCards; | |
| 4815 } | |
| 4816 | |
| 4817 void CMSCollector::checkpointRootsFinal(bool asynch, | |
| 4818 bool clear_all_soft_refs, bool init_mark_was_synchronous) { | |
| 4819 assert(_collectorState == FinalMarking, "incorrect state transition?"); | |
| 4820 check_correct_thread_executing(); | |
| 4821 // world is stopped at this checkpoint | |
| 4822 assert(SafepointSynchronize::is_at_safepoint(), | |
| 4823 "world should be stopped"); | |
|
1703
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
4824 TraceCMSMemoryManagerStats tms(_collectorState); |
| 0 | 4825 verify_work_stacks_empty(); |
| 4826 verify_overflow_empty(); | |
| 4827 | |
| 4828 SpecializationStats::clear(); | |
| 4829 if (PrintGCDetails) { | |
| 4830 gclog_or_tty->print("[YG occupancy: "SIZE_FORMAT" K ("SIZE_FORMAT" K)]", | |
| 4831 _young_gen->used() / K, | |
| 4832 _young_gen->capacity() / K); | |
| 4833 } | |
| 4834 if (asynch) { | |
| 4835 if (CMSScavengeBeforeRemark) { | |
| 4836 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 4837 // Temporarily set flag to false, GCH->do_collection will | |
| 4838 // expect it to be false and set to true | |
| 4839 FlagSetting fl(gch->_is_gc_active, false); | |
| 4840 NOT_PRODUCT(TraceTime t("Scavenge-Before-Remark", | |
| 4841 PrintGCDetails && Verbose, true, gclog_or_tty);) | |
| 4842 int level = _cmsGen->level() - 1; | |
| 4843 if (level >= 0) { | |
| 4844 gch->do_collection(true, // full (i.e. force, see below) | |
| 4845 false, // !clear_all_soft_refs | |
| 4846 0, // size | |
| 4847 false, // is_tlab | |
| 4848 level // max_level | |
| 4849 ); | |
| 4850 } | |
| 4851 } | |
| 4852 FreelistLocker x(this); | |
| 4853 MutexLockerEx y(bitMapLock(), | |
| 4854 Mutex::_no_safepoint_check_flag); | |
| 4855 assert(!init_mark_was_synchronous, "but that's impossible!"); | |
| 4856 checkpointRootsFinalWork(asynch, clear_all_soft_refs, false); | |
| 4857 } else { | |
| 4858 // already have all the locks | |
| 4859 checkpointRootsFinalWork(asynch, clear_all_soft_refs, | |
| 4860 init_mark_was_synchronous); | |
| 4861 } | |
| 4862 verify_work_stacks_empty(); | |
| 4863 verify_overflow_empty(); | |
| 4864 SpecializationStats::print(); | |
| 4865 } | |
| 4866 | |
| 4867 void CMSCollector::checkpointRootsFinalWork(bool asynch, | |
| 4868 bool clear_all_soft_refs, bool init_mark_was_synchronous) { | |
| 4869 | |
| 4870 NOT_PRODUCT(TraceTime tr("checkpointRootsFinalWork", PrintGCDetails, false, gclog_or_tty);) | |
| 4871 | |
| 4872 assert(haveFreelistLocks(), "must have free list locks"); | |
| 4873 assert_lock_strong(bitMapLock()); | |
| 4874 | |
| 4875 if (UseAdaptiveSizePolicy) { | |
| 4876 size_policy()->checkpoint_roots_final_begin(); | |
| 4877 } | |
| 4878 | |
| 4879 ResourceMark rm; | |
| 4880 HandleMark hm; | |
| 4881 | |
| 4882 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 4883 | |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
4884 if (should_unload_classes()) { |
| 0 | 4885 CodeCache::gc_prologue(); |
| 4886 } | |
| 4887 assert(haveFreelistLocks(), "must have free list locks"); | |
| 4888 assert_lock_strong(bitMapLock()); | |
| 4889 | |
|
1190
4788266644c1
6895236: CMS: cmsOopClosures.inline.hpp:43 assert(..., "Should remember klasses in this context")
jmasa
parents:
1145
diff
changeset
|
4890 DEBUG_ONLY(RememberKlassesChecker fmx(should_unload_classes());) |
| 0 | 4891 if (!init_mark_was_synchronous) { |
| 4892 // We might assume that we need not fill TLAB's when | |
| 4893 // CMSScavengeBeforeRemark is set, because we may have just done | |
| 4894 // a scavenge which would have filled all TLAB's -- and besides | |
| 4895 // Eden would be empty. This however may not always be the case -- | |
| 4896 // for instance although we asked for a scavenge, it may not have | |
| 4897 // happened because of a JNI critical section. We probably need | |
| 4898 // a policy for deciding whether we can in that case wait until | |
| 4899 // the critical section releases and then do the remark following | |
| 4900 // the scavenge, and skip it here. In the absence of that policy, | |
| 4901 // or of an indication of whether the scavenge did indeed occur, | |
| 4902 // we cannot rely on TLAB's having been filled and must do | |
| 4903 // so here just in case a scavenge did not happen. | |
| 4904 gch->ensure_parsability(false); // fill TLAB's, but no need to retire them | |
| 4905 // Update the saved marks which may affect the root scans. | |
| 4906 gch->save_marks(); | |
| 4907 | |
| 4908 { | |
| 4909 COMPILER2_PRESENT(DerivedPointerTableDeactivate dpt_deact;) | |
| 4910 | |
| 4911 // Note on the role of the mod union table: | |
| 4912 // Since the marker in "markFromRoots" marks concurrently with | |
| 4913 // mutators, it is possible for some reachable objects not to have been | |
| 4914 // scanned. For instance, an only reference to an object A was | |
| 4915 // placed in object B after the marker scanned B. Unless B is rescanned, | |
| 4916 // A would be collected. Such updates to references in marked objects | |
| 4917 // are detected via the mod union table which is the set of all cards | |
| 4918 // dirtied since the first checkpoint in this GC cycle and prior to | |
| 4919 // the most recent young generation GC, minus those cleaned up by the | |
| 4920 // concurrent precleaning. | |
| 4921 if (CMSParallelRemarkEnabled && ParallelGCThreads > 0) { | |
| 4922 TraceTime t("Rescan (parallel) ", PrintGCDetails, false, gclog_or_tty); | |
| 4923 do_remark_parallel(); | |
| 4924 } else { | |
| 4925 TraceTime t("Rescan (non-parallel) ", PrintGCDetails, false, | |
| 4926 gclog_or_tty); | |
| 4927 do_remark_non_parallel(); | |
| 4928 } | |
| 4929 } | |
| 4930 } else { | |
| 4931 assert(!asynch, "Can't have init_mark_was_synchronous in asynch mode"); | |
| 4932 // The initial mark was stop-world, so there's no rescanning to | |
| 4933 // do; go straight on to the next step below. | |
| 4934 } | |
| 4935 verify_work_stacks_empty(); | |
| 4936 verify_overflow_empty(); | |
| 4937 | |
| 4938 { | |
| 4939 NOT_PRODUCT(TraceTime ts("refProcessingWork", PrintGCDetails, false, gclog_or_tty);) | |
| 4940 refProcessingWork(asynch, clear_all_soft_refs); | |
| 4941 } | |
| 4942 verify_work_stacks_empty(); | |
| 4943 verify_overflow_empty(); | |
| 4944 | |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
4945 if (should_unload_classes()) { |
| 0 | 4946 CodeCache::gc_epilogue(); |
| 4947 } | |
| 4948 | |
| 4949 // If we encountered any (marking stack / work queue) overflow | |
| 4950 // events during the current CMS cycle, take appropriate | |
| 4951 // remedial measures, where possible, so as to try and avoid | |
| 4952 // recurrence of that condition. | |
| 4953 assert(_markStack.isEmpty(), "No grey objects"); | |
| 4954 size_t ser_ovflw = _ser_pmc_remark_ovflw + _ser_pmc_preclean_ovflw + | |
|
452
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
4955 _ser_kac_ovflw + _ser_kac_preclean_ovflw; |
| 0 | 4956 if (ser_ovflw > 0) { |
| 4957 if (PrintCMSStatistics != 0) { | |
| 4958 gclog_or_tty->print_cr("Marking stack overflow (benign) " | |
|
452
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
4959 "(pmc_pc="SIZE_FORMAT", pmc_rm="SIZE_FORMAT", kac="SIZE_FORMAT |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
4960 ", kac_preclean="SIZE_FORMAT")", |
| 0 | 4961 _ser_pmc_preclean_ovflw, _ser_pmc_remark_ovflw, |
|
452
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
4962 _ser_kac_ovflw, _ser_kac_preclean_ovflw); |
| 0 | 4963 } |
| 4964 _markStack.expand(); | |
| 4965 _ser_pmc_remark_ovflw = 0; | |
| 4966 _ser_pmc_preclean_ovflw = 0; | |
|
452
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
4967 _ser_kac_preclean_ovflw = 0; |
| 0 | 4968 _ser_kac_ovflw = 0; |
| 4969 } | |
| 4970 if (_par_pmc_remark_ovflw > 0 || _par_kac_ovflw > 0) { | |
| 4971 if (PrintCMSStatistics != 0) { | |
| 4972 gclog_or_tty->print_cr("Work queue overflow (benign) " | |
| 4973 "(pmc_rm="SIZE_FORMAT", kac="SIZE_FORMAT")", | |
| 4974 _par_pmc_remark_ovflw, _par_kac_ovflw); | |
| 4975 } | |
| 4976 _par_pmc_remark_ovflw = 0; | |
| 4977 _par_kac_ovflw = 0; | |
| 4978 } | |
| 4979 if (PrintCMSStatistics != 0) { | |
| 4980 if (_markStack._hit_limit > 0) { | |
| 4981 gclog_or_tty->print_cr(" (benign) Hit max stack size limit ("SIZE_FORMAT")", | |
| 4982 _markStack._hit_limit); | |
| 4983 } | |
| 4984 if (_markStack._failed_double > 0) { | |
| 4985 gclog_or_tty->print_cr(" (benign) Failed stack doubling ("SIZE_FORMAT")," | |
| 4986 " current capacity "SIZE_FORMAT, | |
| 4987 _markStack._failed_double, | |
| 4988 _markStack.capacity()); | |
| 4989 } | |
| 4990 } | |
| 4991 _markStack._hit_limit = 0; | |
| 4992 _markStack._failed_double = 0; | |
| 4993 | |
| 935 | 4994 // Check that all the klasses have been checked |
| 4995 assert(_revisitStack.isEmpty(), "Not all klasses revisited"); | |
| 4996 | |
| 0 | 4997 if ((VerifyAfterGC || VerifyDuringGC) && |
| 4998 GenCollectedHeap::heap()->total_collections() >= VerifyGCStartAt) { | |
| 4999 verify_after_remark(); | |
| 5000 } | |
| 5001 | |
| 5002 // Change under the freelistLocks. | |
| 5003 _collectorState = Sweeping; | |
| 5004 // Call isAllClear() under bitMapLock | |
| 5005 assert(_modUnionTable.isAllClear(), "Should be clear by end of the" | |
| 5006 " final marking"); | |
| 5007 if (UseAdaptiveSizePolicy) { | |
| 5008 size_policy()->checkpoint_roots_final_end(gch->gc_cause()); | |
| 5009 } | |
| 5010 } | |
| 5011 | |
| 5012 // Parallel remark task | |
| 5013 class CMSParRemarkTask: public AbstractGangTask { | |
| 5014 CMSCollector* _collector; | |
| 5015 WorkGang* _workers; | |
| 5016 int _n_workers; | |
| 5017 CompactibleFreeListSpace* _cms_space; | |
| 5018 CompactibleFreeListSpace* _perm_space; | |
| 5019 | |
| 5020 // The per-thread work queues, available here for stealing. | |
| 5021 OopTaskQueueSet* _task_queues; | |
| 5022 ParallelTaskTerminator _term; | |
| 5023 | |
| 5024 public: | |
| 5025 CMSParRemarkTask(CMSCollector* collector, | |
| 5026 CompactibleFreeListSpace* cms_space, | |
| 5027 CompactibleFreeListSpace* perm_space, | |
| 5028 int n_workers, WorkGang* workers, | |
| 5029 OopTaskQueueSet* task_queues): | |
| 5030 AbstractGangTask("Rescan roots and grey objects in parallel"), | |
| 5031 _collector(collector), | |
| 5032 _cms_space(cms_space), _perm_space(perm_space), | |
| 5033 _n_workers(n_workers), | |
| 5034 _workers(workers), | |
| 5035 _task_queues(task_queues), | |
| 5036 _term(workers->total_workers(), task_queues) { } | |
| 5037 | |
| 5038 OopTaskQueueSet* task_queues() { return _task_queues; } | |
| 5039 | |
| 5040 OopTaskQueue* work_queue(int i) { return task_queues()->queue(i); } | |
| 5041 | |
| 5042 ParallelTaskTerminator* terminator() { return &_term; } | |
| 5043 | |
| 5044 void work(int i); | |
| 5045 | |
| 5046 private: | |
| 5047 // Work method in support of parallel rescan ... of young gen spaces | |
| 5048 void do_young_space_rescan(int i, Par_MarkRefsIntoAndScanClosure* cl, | |
| 5049 ContiguousSpace* space, | |
| 5050 HeapWord** chunk_array, size_t chunk_top); | |
| 5051 | |
| 5052 // ... of dirty cards in old space | |
| 5053 void do_dirty_card_rescan_tasks(CompactibleFreeListSpace* sp, int i, | |
| 5054 Par_MarkRefsIntoAndScanClosure* cl); | |
| 5055 | |
| 5056 // ... work stealing for the above | |
| 5057 void do_work_steal(int i, Par_MarkRefsIntoAndScanClosure* cl, int* seed); | |
| 5058 }; | |
| 5059 | |
| 5060 void CMSParRemarkTask::work(int i) { | |
| 5061 elapsedTimer _timer; | |
| 5062 ResourceMark rm; | |
| 5063 HandleMark hm; | |
| 5064 | |
| 5065 // ---------- rescan from roots -------------- | |
| 5066 _timer.start(); | |
| 5067 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 5068 Par_MarkRefsIntoAndScanClosure par_mrias_cl(_collector, | |
| 5069 _collector->_span, _collector->ref_processor(), | |
| 5070 &(_collector->_markBitMap), | |
| 5071 work_queue(i), &(_collector->_revisitStack)); | |
| 5072 | |
| 5073 // Rescan young gen roots first since these are likely | |
| 5074 // coarsely partitioned and may, on that account, constitute | |
| 5075 // the critical path; thus, it's best to start off that | |
| 5076 // work first. | |
| 5077 // ---------- young gen roots -------------- | |
| 5078 { | |
| 5079 DefNewGeneration* dng = _collector->_young_gen->as_DefNewGeneration(); | |
| 5080 EdenSpace* eden_space = dng->eden(); | |
| 5081 ContiguousSpace* from_space = dng->from(); | |
| 5082 ContiguousSpace* to_space = dng->to(); | |
| 5083 | |
| 5084 HeapWord** eca = _collector->_eden_chunk_array; | |
| 5085 size_t ect = _collector->_eden_chunk_index; | |
| 5086 HeapWord** sca = _collector->_survivor_chunk_array; | |
| 5087 size_t sct = _collector->_survivor_chunk_index; | |
| 5088 | |
| 5089 assert(ect <= _collector->_eden_chunk_capacity, "out of bounds"); | |
| 5090 assert(sct <= _collector->_survivor_chunk_capacity, "out of bounds"); | |
| 5091 | |
| 5092 do_young_space_rescan(i, &par_mrias_cl, to_space, NULL, 0); | |
| 5093 do_young_space_rescan(i, &par_mrias_cl, from_space, sca, sct); | |
| 5094 do_young_space_rescan(i, &par_mrias_cl, eden_space, eca, ect); | |
| 5095 | |
| 5096 _timer.stop(); | |
| 5097 if (PrintCMSStatistics != 0) { | |
| 5098 gclog_or_tty->print_cr( | |
| 5099 "Finished young gen rescan work in %dth thread: %3.3f sec", | |
| 5100 i, _timer.seconds()); | |
| 5101 } | |
| 5102 } | |
| 5103 | |
| 5104 // ---------- remaining roots -------------- | |
| 5105 _timer.reset(); | |
| 5106 _timer.start(); | |
| 5107 gch->gen_process_strong_roots(_collector->_cmsGen->level(), | |
| 5108 false, // yg was scanned above | |
|
989
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
5109 false, // this is parallel code |
| 0 | 5110 true, // collecting perm gen |
| 5111 SharedHeap::ScanningOption(_collector->CMSCollector::roots_scanning_options()), | |
|
989
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
5112 &par_mrias_cl, |
|
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
5113 true, // walk all of code cache if (so & SO_CodeCache) |
|
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
5114 NULL); |
|
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
5115 assert(_collector->should_unload_classes() |
|
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
5116 || (_collector->CMSCollector::roots_scanning_options() & SharedHeap::SO_CodeCache), |
|
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
5117 "if we didn't scan the code cache, we have to be ready to drop nmethods with expired weak oops"); |
| 0 | 5118 _timer.stop(); |
| 5119 if (PrintCMSStatistics != 0) { | |
| 5120 gclog_or_tty->print_cr( | |
| 5121 "Finished remaining root rescan work in %dth thread: %3.3f sec", | |
| 5122 i, _timer.seconds()); | |
| 5123 } | |
| 5124 | |
| 5125 // ---------- rescan dirty cards ------------ | |
| 5126 _timer.reset(); | |
| 5127 _timer.start(); | |
| 5128 | |
| 5129 // Do the rescan tasks for each of the two spaces | |
| 5130 // (cms_space and perm_space) in turn. | |
| 5131 do_dirty_card_rescan_tasks(_cms_space, i, &par_mrias_cl); | |
| 5132 do_dirty_card_rescan_tasks(_perm_space, i, &par_mrias_cl); | |
| 5133 _timer.stop(); | |
| 5134 if (PrintCMSStatistics != 0) { | |
| 5135 gclog_or_tty->print_cr( | |
| 5136 "Finished dirty card rescan work in %dth thread: %3.3f sec", | |
| 5137 i, _timer.seconds()); | |
| 5138 } | |
| 5139 | |
| 5140 // ---------- steal work from other threads ... | |
| 5141 // ---------- ... and drain overflow list. | |
| 5142 _timer.reset(); | |
| 5143 _timer.start(); | |
| 5144 do_work_steal(i, &par_mrias_cl, _collector->hash_seed(i)); | |
| 5145 _timer.stop(); | |
| 5146 if (PrintCMSStatistics != 0) { | |
| 5147 gclog_or_tty->print_cr( | |
| 5148 "Finished work stealing in %dth thread: %3.3f sec", | |
| 5149 i, _timer.seconds()); | |
| 5150 } | |
| 5151 } | |
| 5152 | |
| 5153 void | |
| 5154 CMSParRemarkTask::do_young_space_rescan(int i, | |
| 5155 Par_MarkRefsIntoAndScanClosure* cl, ContiguousSpace* space, | |
| 5156 HeapWord** chunk_array, size_t chunk_top) { | |
| 5157 // Until all tasks completed: | |
| 5158 // . claim an unclaimed task | |
| 5159 // . compute region boundaries corresponding to task claimed | |
| 5160 // using chunk_array | |
| 5161 // . par_oop_iterate(cl) over that region | |
| 5162 | |
| 5163 ResourceMark rm; | |
| 5164 HandleMark hm; | |
| 5165 | |
| 5166 SequentialSubTasksDone* pst = space->par_seq_tasks(); | |
| 5167 assert(pst->valid(), "Uninitialized use?"); | |
| 5168 | |
| 5169 int nth_task = 0; | |
| 5170 int n_tasks = pst->n_tasks(); | |
| 5171 | |
| 5172 HeapWord *start, *end; | |
| 5173 while (!pst->is_task_claimed(/* reference */ nth_task)) { | |
| 5174 // We claimed task # nth_task; compute its boundaries. | |
| 5175 if (chunk_top == 0) { // no samples were taken | |
| 5176 assert(nth_task == 0 && n_tasks == 1, "Can have only 1 EdenSpace task"); | |
| 5177 start = space->bottom(); | |
| 5178 end = space->top(); | |
| 5179 } else if (nth_task == 0) { | |
| 5180 start = space->bottom(); | |
| 5181 end = chunk_array[nth_task]; | |
| 5182 } else if (nth_task < (jint)chunk_top) { | |
| 5183 assert(nth_task >= 1, "Control point invariant"); | |
| 5184 start = chunk_array[nth_task - 1]; | |
| 5185 end = chunk_array[nth_task]; | |
| 5186 } else { | |
| 5187 assert(nth_task == (jint)chunk_top, "Control point invariant"); | |
| 5188 start = chunk_array[chunk_top - 1]; | |
| 5189 end = space->top(); | |
| 5190 } | |
| 5191 MemRegion mr(start, end); | |
| 5192 // Verify that mr is in space | |
| 5193 assert(mr.is_empty() || space->used_region().contains(mr), | |
| 5194 "Should be in space"); | |
| 5195 // Verify that "start" is an object boundary | |
| 5196 assert(mr.is_empty() || oop(mr.start())->is_oop(), | |
| 5197 "Should be an oop"); | |
| 5198 space->par_oop_iterate(mr, cl); | |
| 5199 } | |
| 5200 pst->all_tasks_completed(); | |
| 5201 } | |
| 5202 | |
| 5203 void | |
| 5204 CMSParRemarkTask::do_dirty_card_rescan_tasks( | |
| 5205 CompactibleFreeListSpace* sp, int i, | |
| 5206 Par_MarkRefsIntoAndScanClosure* cl) { | |
| 5207 // Until all tasks completed: | |
| 5208 // . claim an unclaimed task | |
| 5209 // . compute region boundaries corresponding to task claimed | |
| 5210 // . transfer dirty bits ct->mut for that region | |
| 5211 // . apply rescanclosure to dirty mut bits for that region | |
| 5212 | |
| 5213 ResourceMark rm; | |
| 5214 HandleMark hm; | |
| 5215 | |
| 5216 OopTaskQueue* work_q = work_queue(i); | |
| 5217 ModUnionClosure modUnionClosure(&(_collector->_modUnionTable)); | |
| 5218 // CAUTION! CAUTION! CAUTION! CAUTION! CAUTION! CAUTION! CAUTION! | |
| 5219 // CAUTION: This closure has state that persists across calls to | |
| 5220 // the work method dirty_range_iterate_clear() in that it has | |
| 5221 // imbedded in it a (subtype of) UpwardsObjectClosure. The | |
| 5222 // use of that state in the imbedded UpwardsObjectClosure instance | |
| 5223 // assumes that the cards are always iterated (even if in parallel | |
| 5224 // by several threads) in monotonically increasing order per each | |
| 5225 // thread. This is true of the implementation below which picks | |
| 5226 // card ranges (chunks) in monotonically increasing order globally | |
| 5227 // and, a-fortiori, in monotonically increasing order per thread | |
| 5228 // (the latter order being a subsequence of the former). | |
| 5229 // If the work code below is ever reorganized into a more chaotic | |
| 5230 // work-partitioning form than the current "sequential tasks" | |
| 5231 // paradigm, the use of that persistent state will have to be | |
| 5232 // revisited and modified appropriately. See also related | |
| 5233 // bug 4756801 work on which should examine this code to make | |
| 5234 // sure that the changes there do not run counter to the | |
| 5235 // assumptions made here and necessary for correctness and | |
| 5236 // efficiency. Note also that this code might yield inefficient | |
| 5237 // behaviour in the case of very large objects that span one or | |
| 5238 // more work chunks. Such objects would potentially be scanned | |
| 5239 // several times redundantly. Work on 4756801 should try and | |
| 5240 // address that performance anomaly if at all possible. XXX | |
| 5241 MemRegion full_span = _collector->_span; | |
| 5242 CMSBitMap* bm = &(_collector->_markBitMap); // shared | |
| 5243 CMSMarkStack* rs = &(_collector->_revisitStack); // shared | |
| 5244 MarkFromDirtyCardsClosure | |
| 5245 greyRescanClosure(_collector, full_span, // entire span of interest | |
| 5246 sp, bm, work_q, rs, cl); | |
| 5247 | |
| 5248 SequentialSubTasksDone* pst = sp->conc_par_seq_tasks(); | |
| 5249 assert(pst->valid(), "Uninitialized use?"); | |
| 5250 int nth_task = 0; | |
| 5251 const int alignment = CardTableModRefBS::card_size * BitsPerWord; | |
| 5252 MemRegion span = sp->used_region(); | |
| 5253 HeapWord* start_addr = span.start(); | |
| 5254 HeapWord* end_addr = (HeapWord*)round_to((intptr_t)span.end(), | |
| 5255 alignment); | |
| 5256 const size_t chunk_size = sp->rescan_task_size(); // in HeapWord units | |
| 5257 assert((HeapWord*)round_to((intptr_t)start_addr, alignment) == | |
| 5258 start_addr, "Check alignment"); | |
| 5259 assert((size_t)round_to((intptr_t)chunk_size, alignment) == | |
| 5260 chunk_size, "Check alignment"); | |
| 5261 | |
| 5262 while (!pst->is_task_claimed(/* reference */ nth_task)) { | |
| 5263 // Having claimed the nth_task, compute corresponding mem-region, | |
| 5264 // which is a-fortiori aligned correctly (i.e. at a MUT bopundary). | |
| 5265 // The alignment restriction ensures that we do not need any | |
| 5266 // synchronization with other gang-workers while setting or | |
| 5267 // clearing bits in thus chunk of the MUT. | |
| 5268 MemRegion this_span = MemRegion(start_addr + nth_task*chunk_size, | |
| 5269 start_addr + (nth_task+1)*chunk_size); | |
| 5270 // The last chunk's end might be way beyond end of the | |
| 5271 // used region. In that case pull back appropriately. | |
| 5272 if (this_span.end() > end_addr) { | |
| 5273 this_span.set_end(end_addr); | |
| 5274 assert(!this_span.is_empty(), "Program logic (calculation of n_tasks)"); | |
| 5275 } | |
| 5276 // Iterate over the dirty cards covering this chunk, marking them | |
| 5277 // precleaned, and setting the corresponding bits in the mod union | |
| 5278 // table. Since we have been careful to partition at Card and MUT-word | |
| 5279 // boundaries no synchronization is needed between parallel threads. | |
| 5280 _collector->_ct->ct_bs()->dirty_card_iterate(this_span, | |
| 5281 &modUnionClosure); | |
| 5282 | |
| 5283 // Having transferred these marks into the modUnionTable, | |
| 5284 // rescan the marked objects on the dirty cards in the modUnionTable. | |
| 5285 // Even if this is at a synchronous collection, the initial marking | |
| 5286 // may have been done during an asynchronous collection so there | |
| 5287 // may be dirty bits in the mod-union table. | |
| 5288 _collector->_modUnionTable.dirty_range_iterate_clear( | |
| 5289 this_span, &greyRescanClosure); | |
| 5290 _collector->_modUnionTable.verifyNoOneBitsInRange( | |
| 5291 this_span.start(), | |
| 5292 this_span.end()); | |
| 5293 } | |
| 5294 pst->all_tasks_completed(); // declare that i am done | |
| 5295 } | |
| 5296 | |
| 5297 // . see if we can share work_queues with ParNew? XXX | |
| 5298 void | |
| 5299 CMSParRemarkTask::do_work_steal(int i, Par_MarkRefsIntoAndScanClosure* cl, | |
| 5300 int* seed) { | |
| 5301 OopTaskQueue* work_q = work_queue(i); | |
| 5302 NOT_PRODUCT(int num_steals = 0;) | |
| 5303 oop obj_to_scan; | |
| 5304 CMSBitMap* bm = &(_collector->_markBitMap); | |
| 5305 | |
| 5306 while (true) { | |
| 5307 // Completely finish any left over work from (an) earlier round(s) | |
| 5308 cl->trim_queue(0); | |
|
679
cea947c8a988
6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops
ysr
parents:
628
diff
changeset
|
5309 size_t num_from_overflow_list = MIN2((size_t)(work_q->max_elems() - work_q->size())/4, |
|
cea947c8a988
6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops
ysr
parents:
628
diff
changeset
|
5310 (size_t)ParGCDesiredObjsFromOverflowList); |
| 0 | 5311 // Now check if there's any work in the overflow list |
| 5312 if (_collector->par_take_from_overflow_list(num_from_overflow_list, | |
| 5313 work_q)) { | |
| 5314 // found something in global overflow list; | |
| 5315 // not yet ready to go stealing work from others. | |
| 5316 // We'd like to assert(work_q->size() != 0, ...) | |
| 5317 // because we just took work from the overflow list, | |
| 5318 // but of course we can't since all of that could have | |
| 5319 // been already stolen from us. | |
| 5320 // "He giveth and He taketh away." | |
| 5321 continue; | |
| 5322 } | |
| 5323 // Verify that we have no work before we resort to stealing | |
| 5324 assert(work_q->size() == 0, "Have work, shouldn't steal"); | |
| 5325 // Try to steal from other queues that have work | |
| 5326 if (task_queues()->steal(i, seed, /* reference */ obj_to_scan)) { | |
| 5327 NOT_PRODUCT(num_steals++;) | |
| 5328 assert(obj_to_scan->is_oop(), "Oops, not an oop!"); | |
| 5329 assert(bm->isMarked((HeapWord*)obj_to_scan), "Stole an unmarked oop?"); | |
| 5330 // Do scanning work | |
| 5331 obj_to_scan->oop_iterate(cl); | |
| 5332 // Loop around, finish this work, and try to steal some more | |
| 5333 } else if (terminator()->offer_termination()) { | |
| 5334 break; // nirvana from the infinite cycle | |
| 5335 } | |
| 5336 } | |
| 5337 NOT_PRODUCT( | |
| 5338 if (PrintCMSStatistics != 0) { | |
| 5339 gclog_or_tty->print("\n\t(%d: stole %d oops)", i, num_steals); | |
| 5340 } | |
| 5341 ) | |
| 5342 assert(work_q->size() == 0 && _collector->overflow_list_is_empty(), | |
| 5343 "Else our work is not yet done"); | |
| 5344 } | |
| 5345 | |
| 5346 // Return a thread-local PLAB recording array, as appropriate. | |
| 5347 void* CMSCollector::get_data_recorder(int thr_num) { | |
| 5348 if (_survivor_plab_array != NULL && | |
| 5349 (CMSPLABRecordAlways || | |
| 5350 (_collectorState > Marking && _collectorState < FinalMarking))) { | |
| 5351 assert(thr_num < (int)ParallelGCThreads, "thr_num is out of bounds"); | |
| 5352 ChunkArray* ca = &_survivor_plab_array[thr_num]; | |
| 5353 ca->reset(); // clear it so that fresh data is recorded | |
| 5354 return (void*) ca; | |
| 5355 } else { | |
| 5356 return NULL; | |
| 5357 } | |
| 5358 } | |
| 5359 | |
| 5360 // Reset all the thread-local PLAB recording arrays | |
| 5361 void CMSCollector::reset_survivor_plab_arrays() { | |
| 5362 for (uint i = 0; i < ParallelGCThreads; i++) { | |
| 5363 _survivor_plab_array[i].reset(); | |
| 5364 } | |
| 5365 } | |
| 5366 | |
| 5367 // Merge the per-thread plab arrays into the global survivor chunk | |
| 5368 // array which will provide the partitioning of the survivor space | |
| 5369 // for CMS rescan. | |
| 5370 void CMSCollector::merge_survivor_plab_arrays(ContiguousSpace* surv) { | |
| 5371 assert(_survivor_plab_array != NULL, "Error"); | |
| 5372 assert(_survivor_chunk_array != NULL, "Error"); | |
| 5373 assert(_collectorState == FinalMarking, "Error"); | |
| 5374 for (uint j = 0; j < ParallelGCThreads; j++) { | |
| 5375 _cursor[j] = 0; | |
| 5376 } | |
| 5377 HeapWord* top = surv->top(); | |
| 5378 size_t i; | |
| 5379 for (i = 0; i < _survivor_chunk_capacity; i++) { // all sca entries | |
| 5380 HeapWord* min_val = top; // Higher than any PLAB address | |
| 5381 uint min_tid = 0; // position of min_val this round | |
| 5382 for (uint j = 0; j < ParallelGCThreads; j++) { | |
| 5383 ChunkArray* cur_sca = &_survivor_plab_array[j]; | |
| 5384 if (_cursor[j] == cur_sca->end()) { | |
| 5385 continue; | |
| 5386 } | |
| 5387 assert(_cursor[j] < cur_sca->end(), "ctl pt invariant"); | |
| 5388 HeapWord* cur_val = cur_sca->nth(_cursor[j]); | |
| 5389 assert(surv->used_region().contains(cur_val), "Out of bounds value"); | |
| 5390 if (cur_val < min_val) { | |
| 5391 min_tid = j; | |
| 5392 min_val = cur_val; | |
| 5393 } else { | |
| 5394 assert(cur_val < top, "All recorded addresses should be less"); | |
| 5395 } | |
| 5396 } | |
| 5397 // At this point min_val and min_tid are respectively | |
| 5398 // the least address in _survivor_plab_array[j]->nth(_cursor[j]) | |
| 5399 // and the thread (j) that witnesses that address. | |
| 5400 // We record this address in the _survivor_chunk_array[i] | |
| 5401 // and increment _cursor[min_tid] prior to the next round i. | |
| 5402 if (min_val == top) { | |
| 5403 break; | |
| 5404 } | |
| 5405 _survivor_chunk_array[i] = min_val; | |
| 5406 _cursor[min_tid]++; | |
| 5407 } | |
| 5408 // We are all done; record the size of the _survivor_chunk_array | |
| 5409 _survivor_chunk_index = i; // exclusive: [0, i) | |
| 5410 if (PrintCMSStatistics > 0) { | |
| 5411 gclog_or_tty->print(" (Survivor:" SIZE_FORMAT "chunks) ", i); | |
| 5412 } | |
| 5413 // Verify that we used up all the recorded entries | |
| 5414 #ifdef ASSERT | |
| 5415 size_t total = 0; | |
| 5416 for (uint j = 0; j < ParallelGCThreads; j++) { | |
| 5417 assert(_cursor[j] == _survivor_plab_array[j].end(), "Ctl pt invariant"); | |
| 5418 total += _cursor[j]; | |
| 5419 } | |
| 5420 assert(total == _survivor_chunk_index, "Ctl Pt Invariant"); | |
| 5421 // Check that the merged array is in sorted order | |
| 5422 if (total > 0) { | |
| 5423 for (size_t i = 0; i < total - 1; i++) { | |
| 5424 if (PrintCMSStatistics > 0) { | |
| 5425 gclog_or_tty->print(" (chunk" SIZE_FORMAT ":" INTPTR_FORMAT ") ", | |
| 5426 i, _survivor_chunk_array[i]); | |
| 5427 } | |
| 5428 assert(_survivor_chunk_array[i] < _survivor_chunk_array[i+1], | |
| 5429 "Not sorted"); | |
| 5430 } | |
| 5431 } | |
| 5432 #endif // ASSERT | |
| 5433 } | |
| 5434 | |
| 5435 // Set up the space's par_seq_tasks structure for work claiming | |
| 5436 // for parallel rescan of young gen. | |
| 5437 // See ParRescanTask where this is currently used. | |
| 5438 void | |
| 5439 CMSCollector:: | |
| 5440 initialize_sequential_subtasks_for_young_gen_rescan(int n_threads) { | |
| 5441 assert(n_threads > 0, "Unexpected n_threads argument"); | |
| 5442 DefNewGeneration* dng = (DefNewGeneration*)_young_gen; | |
| 5443 | |
| 5444 // Eden space | |
| 5445 { | |
| 5446 SequentialSubTasksDone* pst = dng->eden()->par_seq_tasks(); | |
| 5447 assert(!pst->valid(), "Clobbering existing data?"); | |
| 5448 // Each valid entry in [0, _eden_chunk_index) represents a task. | |
| 5449 size_t n_tasks = _eden_chunk_index + 1; | |
| 5450 assert(n_tasks == 1 || _eden_chunk_array != NULL, "Error"); | |
| 5451 pst->set_par_threads(n_threads); | |
| 5452 pst->set_n_tasks((int)n_tasks); | |
| 5453 } | |
| 5454 | |
| 5455 // Merge the survivor plab arrays into _survivor_chunk_array | |
| 5456 if (_survivor_plab_array != NULL) { | |
| 5457 merge_survivor_plab_arrays(dng->from()); | |
| 5458 } else { | |
| 5459 assert(_survivor_chunk_index == 0, "Error"); | |
| 5460 } | |
| 5461 | |
| 5462 // To space | |
| 5463 { | |
| 5464 SequentialSubTasksDone* pst = dng->to()->par_seq_tasks(); | |
| 5465 assert(!pst->valid(), "Clobbering existing data?"); | |
| 5466 pst->set_par_threads(n_threads); | |
| 5467 pst->set_n_tasks(1); | |
| 5468 assert(pst->valid(), "Error"); | |
| 5469 } | |
| 5470 | |
| 5471 // From space | |
| 5472 { | |
| 5473 SequentialSubTasksDone* pst = dng->from()->par_seq_tasks(); | |
| 5474 assert(!pst->valid(), "Clobbering existing data?"); | |
| 5475 size_t n_tasks = _survivor_chunk_index + 1; | |
| 5476 assert(n_tasks == 1 || _survivor_chunk_array != NULL, "Error"); | |
| 5477 pst->set_par_threads(n_threads); | |
| 5478 pst->set_n_tasks((int)n_tasks); | |
| 5479 assert(pst->valid(), "Error"); | |
| 5480 } | |
| 5481 } | |
| 5482 | |
| 5483 // Parallel version of remark | |
| 5484 void CMSCollector::do_remark_parallel() { | |
| 5485 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 5486 WorkGang* workers = gch->workers(); | |
| 5487 assert(workers != NULL, "Need parallel worker threads."); | |
| 5488 int n_workers = workers->total_workers(); | |
| 5489 CompactibleFreeListSpace* cms_space = _cmsGen->cmsSpace(); | |
| 5490 CompactibleFreeListSpace* perm_space = _permGen->cmsSpace(); | |
| 5491 | |
| 5492 CMSParRemarkTask tsk(this, | |
| 5493 cms_space, perm_space, | |
| 5494 n_workers, workers, task_queues()); | |
| 5495 | |
| 5496 // Set up for parallel process_strong_roots work. | |
| 5497 gch->set_par_threads(n_workers); | |
| 5498 // We won't be iterating over the cards in the card table updating | |
| 5499 // the younger_gen cards, so we shouldn't call the following else | |
| 5500 // the verification code as well as subsequent younger_refs_iterate | |
| 5501 // code would get confused. XXX | |
| 5502 // gch->rem_set()->prepare_for_younger_refs_iterate(true); // parallel | |
| 5503 | |
| 5504 // The young gen rescan work will not be done as part of | |
| 5505 // process_strong_roots (which currently doesn't knw how to | |
| 5506 // parallelize such a scan), but rather will be broken up into | |
| 5507 // a set of parallel tasks (via the sampling that the [abortable] | |
| 5508 // preclean phase did of EdenSpace, plus the [two] tasks of | |
| 5509 // scanning the [two] survivor spaces. Further fine-grain | |
| 5510 // parallelization of the scanning of the survivor spaces | |
| 5511 // themselves, and of precleaning of the younger gen itself | |
| 5512 // is deferred to the future. | |
| 5513 initialize_sequential_subtasks_for_young_gen_rescan(n_workers); | |
| 5514 | |
| 5515 // The dirty card rescan work is broken up into a "sequence" | |
| 5516 // of parallel tasks (per constituent space) that are dynamically | |
| 5517 // claimed by the parallel threads. | |
| 5518 cms_space->initialize_sequential_subtasks_for_rescan(n_workers); | |
| 5519 perm_space->initialize_sequential_subtasks_for_rescan(n_workers); | |
| 5520 | |
| 5521 // It turns out that even when we're using 1 thread, doing the work in a | |
| 5522 // separate thread causes wide variance in run times. We can't help this | |
| 5523 // in the multi-threaded case, but we special-case n=1 here to get | |
| 5524 // repeatable measurements of the 1-thread overhead of the parallel code. | |
| 5525 if (n_workers > 1) { | |
| 5526 // Make refs discovery MT-safe | |
| 5527 ReferenceProcessorMTMutator mt(ref_processor(), true); | |
|
989
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
5528 GenCollectedHeap::StrongRootsScope srs(gch); |
| 0 | 5529 workers->run_task(&tsk); |
| 5530 } else { | |
|
989
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
5531 GenCollectedHeap::StrongRootsScope srs(gch); |
| 0 | 5532 tsk.work(0); |
| 5533 } | |
| 5534 gch->set_par_threads(0); // 0 ==> non-parallel. | |
| 5535 // restore, single-threaded for now, any preserved marks | |
| 5536 // as a result of work_q overflow | |
| 5537 restore_preserved_marks_if_any(); | |
| 5538 } | |
| 5539 | |
| 5540 // Non-parallel version of remark | |
| 5541 void CMSCollector::do_remark_non_parallel() { | |
| 5542 ResourceMark rm; | |
| 5543 HandleMark hm; | |
| 5544 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 5545 MarkRefsIntoAndScanClosure | |
| 5546 mrias_cl(_span, ref_processor(), &_markBitMap, &_modUnionTable, | |
| 5547 &_markStack, &_revisitStack, this, | |
| 5548 false /* should_yield */, false /* not precleaning */); | |
| 5549 MarkFromDirtyCardsClosure | |
| 5550 markFromDirtyCardsClosure(this, _span, | |
| 5551 NULL, // space is set further below | |
| 5552 &_markBitMap, &_markStack, &_revisitStack, | |
| 5553 &mrias_cl); | |
| 5554 { | |
| 5555 TraceTime t("grey object rescan", PrintGCDetails, false, gclog_or_tty); | |
|
342
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
5556 // Iterate over the dirty cards, setting the corresponding bits in the |
|
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
5557 // mod union table. |
| 0 | 5558 { |
| 5559 ModUnionClosure modUnionClosure(&_modUnionTable); | |
| 5560 _ct->ct_bs()->dirty_card_iterate( | |
| 5561 _cmsGen->used_region(), | |
| 5562 &modUnionClosure); | |
| 5563 _ct->ct_bs()->dirty_card_iterate( | |
| 5564 _permGen->used_region(), | |
| 5565 &modUnionClosure); | |
| 5566 } | |
| 5567 // Having transferred these marks into the modUnionTable, we just need | |
| 5568 // to rescan the marked objects on the dirty cards in the modUnionTable. | |
| 5569 // The initial marking may have been done during an asynchronous | |
| 5570 // collection so there may be dirty bits in the mod-union table. | |
| 5571 const int alignment = | |
| 5572 CardTableModRefBS::card_size * BitsPerWord; | |
| 5573 { | |
| 5574 // ... First handle dirty cards in CMS gen | |
| 5575 markFromDirtyCardsClosure.set_space(_cmsGen->cmsSpace()); | |
| 5576 MemRegion ur = _cmsGen->used_region(); | |
| 5577 HeapWord* lb = ur.start(); | |
| 5578 HeapWord* ub = (HeapWord*)round_to((intptr_t)ur.end(), alignment); | |
| 5579 MemRegion cms_span(lb, ub); | |
| 5580 _modUnionTable.dirty_range_iterate_clear(cms_span, | |
| 5581 &markFromDirtyCardsClosure); | |
| 5582 verify_work_stacks_empty(); | |
| 5583 if (PrintCMSStatistics != 0) { | |
| 5584 gclog_or_tty->print(" (re-scanned "SIZE_FORMAT" dirty cards in cms gen) ", | |
| 5585 markFromDirtyCardsClosure.num_dirty_cards()); | |
| 5586 } | |
| 5587 } | |
| 5588 { | |
| 5589 // .. and then repeat for dirty cards in perm gen | |
| 5590 markFromDirtyCardsClosure.set_space(_permGen->cmsSpace()); | |
| 5591 MemRegion ur = _permGen->used_region(); | |
| 5592 HeapWord* lb = ur.start(); | |
| 5593 HeapWord* ub = (HeapWord*)round_to((intptr_t)ur.end(), alignment); | |
| 5594 MemRegion perm_span(lb, ub); | |
| 5595 _modUnionTable.dirty_range_iterate_clear(perm_span, | |
| 5596 &markFromDirtyCardsClosure); | |
| 5597 verify_work_stacks_empty(); | |
| 5598 if (PrintCMSStatistics != 0) { | |
| 5599 gclog_or_tty->print(" (re-scanned "SIZE_FORMAT" dirty cards in perm gen) ", | |
| 5600 markFromDirtyCardsClosure.num_dirty_cards()); | |
| 5601 } | |
| 5602 } | |
| 5603 } | |
| 5604 if (VerifyDuringGC && | |
| 5605 GenCollectedHeap::heap()->total_collections() >= VerifyGCStartAt) { | |
| 5606 HandleMark hm; // Discard invalid handles created during verification | |
| 5607 Universe::verify(true); | |
| 5608 } | |
| 5609 { | |
| 5610 TraceTime t("root rescan", PrintGCDetails, false, gclog_or_tty); | |
| 5611 | |
| 5612 verify_work_stacks_empty(); | |
| 5613 | |
| 5614 gch->rem_set()->prepare_for_younger_refs_iterate(false); // Not parallel. | |
|
989
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
5615 GenCollectedHeap::StrongRootsScope srs(gch); |
| 0 | 5616 gch->gen_process_strong_roots(_cmsGen->level(), |
| 5617 true, // younger gens as roots | |
|
989
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
5618 false, // use the local StrongRootsScope |
| 0 | 5619 true, // collecting perm gen |
| 5620 SharedHeap::ScanningOption(roots_scanning_options()), | |
|
989
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
5621 &mrias_cl, |
|
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
5622 true, // walk code active on stacks |
|
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
5623 NULL); |
|
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
5624 assert(should_unload_classes() |
|
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
5625 || (roots_scanning_options() & SharedHeap::SO_CodeCache), |
|
148e5441d916
6863023: need non-perm oops in code cache for JSR 292
jrose
parents:
798
diff
changeset
|
5626 "if we didn't scan the code cache, we have to be ready to drop nmethods with expired weak oops"); |
| 0 | 5627 } |
| 5628 verify_work_stacks_empty(); | |
| 5629 // Restore evacuated mark words, if any, used for overflow list links | |
| 5630 if (!CMSOverflowEarlyRestoration) { | |
| 5631 restore_preserved_marks_if_any(); | |
| 5632 } | |
| 5633 verify_overflow_empty(); | |
| 5634 } | |
| 5635 | |
| 5636 //////////////////////////////////////////////////////// | |
| 5637 // Parallel Reference Processing Task Proxy Class | |
| 5638 //////////////////////////////////////////////////////// | |
| 5639 class CMSRefProcTaskProxy: public AbstractGangTask { | |
| 5640 typedef AbstractRefProcTaskExecutor::ProcessTask ProcessTask; | |
| 5641 CMSCollector* _collector; | |
| 5642 CMSBitMap* _mark_bit_map; | |
|
143
b5489bb705c9
6662086: 6u4+, 7b11+: CMS never clears referents when -XX:+ParallelRefProcEnabled
ysr
parents:
113
diff
changeset
|
5643 const MemRegion _span; |
| 0 | 5644 OopTaskQueueSet* _task_queues; |
| 5645 ParallelTaskTerminator _term; | |
| 5646 ProcessTask& _task; | |
| 5647 | |
| 5648 public: | |
| 5649 CMSRefProcTaskProxy(ProcessTask& task, | |
| 5650 CMSCollector* collector, | |
| 5651 const MemRegion& span, | |
| 5652 CMSBitMap* mark_bit_map, | |
| 5653 int total_workers, | |
| 5654 OopTaskQueueSet* task_queues): | |
| 5655 AbstractGangTask("Process referents by policy in parallel"), | |
| 5656 _task(task), | |
| 5657 _collector(collector), _span(span), _mark_bit_map(mark_bit_map), | |
| 5658 _task_queues(task_queues), | |
| 5659 _term(total_workers, task_queues) | |
|
143
b5489bb705c9
6662086: 6u4+, 7b11+: CMS never clears referents when -XX:+ParallelRefProcEnabled
ysr
parents:
113
diff
changeset
|
5660 { |
|
b5489bb705c9
6662086: 6u4+, 7b11+: CMS never clears referents when -XX:+ParallelRefProcEnabled
ysr
parents:
113
diff
changeset
|
5661 assert(_collector->_span.equals(_span) && !_span.is_empty(), |
|
b5489bb705c9
6662086: 6u4+, 7b11+: CMS never clears referents when -XX:+ParallelRefProcEnabled
ysr
parents:
113
diff
changeset
|
5662 "Inconsistency in _span"); |
|
b5489bb705c9
6662086: 6u4+, 7b11+: CMS never clears referents when -XX:+ParallelRefProcEnabled
ysr
parents:
113
diff
changeset
|
5663 } |
| 0 | 5664 |
| 5665 OopTaskQueueSet* task_queues() { return _task_queues; } | |
| 5666 | |
| 5667 OopTaskQueue* work_queue(int i) { return task_queues()->queue(i); } | |
| 5668 | |
| 5669 ParallelTaskTerminator* terminator() { return &_term; } | |
| 5670 | |
| 5671 void do_work_steal(int i, | |
| 5672 CMSParDrainMarkingStackClosure* drain, | |
| 5673 CMSParKeepAliveClosure* keep_alive, | |
| 5674 int* seed); | |
| 5675 | |
| 5676 virtual void work(int i); | |
| 5677 }; | |
| 5678 | |
| 5679 void CMSRefProcTaskProxy::work(int i) { | |
|
143
b5489bb705c9
6662086: 6u4+, 7b11+: CMS never clears referents when -XX:+ParallelRefProcEnabled
ysr
parents:
113
diff
changeset
|
5680 assert(_collector->_span.equals(_span), "Inconsistency in _span"); |
| 0 | 5681 CMSParKeepAliveClosure par_keep_alive(_collector, _span, |
| 935 | 5682 _mark_bit_map, |
| 5683 &_collector->_revisitStack, | |
| 5684 work_queue(i)); | |
| 0 | 5685 CMSParDrainMarkingStackClosure par_drain_stack(_collector, _span, |
| 935 | 5686 _mark_bit_map, |
| 5687 &_collector->_revisitStack, | |
| 5688 work_queue(i)); | |
|
143
b5489bb705c9
6662086: 6u4+, 7b11+: CMS never clears referents when -XX:+ParallelRefProcEnabled
ysr
parents:
113
diff
changeset
|
5689 CMSIsAliveClosure is_alive_closure(_span, _mark_bit_map); |
| 0 | 5690 _task.work(i, is_alive_closure, par_keep_alive, par_drain_stack); |
| 5691 if (_task.marks_oops_alive()) { | |
| 5692 do_work_steal(i, &par_drain_stack, &par_keep_alive, | |
| 5693 _collector->hash_seed(i)); | |
| 5694 } | |
| 5695 assert(work_queue(i)->size() == 0, "work_queue should be empty"); | |
| 5696 assert(_collector->_overflow_list == NULL, "non-empty _overflow_list"); | |
| 5697 } | |
| 5698 | |
| 5699 class CMSRefEnqueueTaskProxy: public AbstractGangTask { | |
| 5700 typedef AbstractRefProcTaskExecutor::EnqueueTask EnqueueTask; | |
| 5701 EnqueueTask& _task; | |
| 5702 | |
| 5703 public: | |
| 5704 CMSRefEnqueueTaskProxy(EnqueueTask& task) | |
| 5705 : AbstractGangTask("Enqueue reference objects in parallel"), | |
| 5706 _task(task) | |
| 5707 { } | |
| 5708 | |
| 5709 virtual void work(int i) | |
| 5710 { | |
| 5711 _task.work(i); | |
| 5712 } | |
| 5713 }; | |
| 5714 | |
| 5715 CMSParKeepAliveClosure::CMSParKeepAliveClosure(CMSCollector* collector, | |
| 935 | 5716 MemRegion span, CMSBitMap* bit_map, CMSMarkStack* revisit_stack, |
| 5717 OopTaskQueue* work_queue): | |
| 5718 Par_KlassRememberingOopClosure(collector, NULL, revisit_stack), | |
| 0 | 5719 _span(span), |
| 5720 _bit_map(bit_map), | |
| 5721 _work_queue(work_queue), | |
| 935 | 5722 _mark_and_push(collector, span, bit_map, revisit_stack, work_queue), |
| 0 | 5723 _low_water_mark(MIN2((uint)(work_queue->max_elems()/4), |
| 5724 (uint)(CMSWorkQueueDrainThreshold * ParallelGCThreads))) | |
| 5725 { } | |
| 5726 | |
| 5727 // . see if we can share work_queues with ParNew? XXX | |
| 5728 void CMSRefProcTaskProxy::do_work_steal(int i, | |
| 5729 CMSParDrainMarkingStackClosure* drain, | |
| 5730 CMSParKeepAliveClosure* keep_alive, | |
| 5731 int* seed) { | |
| 5732 OopTaskQueue* work_q = work_queue(i); | |
| 5733 NOT_PRODUCT(int num_steals = 0;) | |
| 5734 oop obj_to_scan; | |
| 5735 | |
| 5736 while (true) { | |
| 5737 // Completely finish any left over work from (an) earlier round(s) | |
| 5738 drain->trim_queue(0); | |
|
679
cea947c8a988
6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops
ysr
parents:
628
diff
changeset
|
5739 size_t num_from_overflow_list = MIN2((size_t)(work_q->max_elems() - work_q->size())/4, |
|
cea947c8a988
6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops
ysr
parents:
628
diff
changeset
|
5740 (size_t)ParGCDesiredObjsFromOverflowList); |
| 0 | 5741 // Now check if there's any work in the overflow list |
| 5742 if (_collector->par_take_from_overflow_list(num_from_overflow_list, | |
| 5743 work_q)) { | |
| 5744 // Found something in global overflow list; | |
| 5745 // not yet ready to go stealing work from others. | |
| 5746 // We'd like to assert(work_q->size() != 0, ...) | |
| 5747 // because we just took work from the overflow list, | |
| 5748 // but of course we can't, since all of that might have | |
| 5749 // been already stolen from us. | |
| 5750 continue; | |
| 5751 } | |
| 5752 // Verify that we have no work before we resort to stealing | |
| 5753 assert(work_q->size() == 0, "Have work, shouldn't steal"); | |
| 5754 // Try to steal from other queues that have work | |
| 5755 if (task_queues()->steal(i, seed, /* reference */ obj_to_scan)) { | |
| 5756 NOT_PRODUCT(num_steals++;) | |
| 5757 assert(obj_to_scan->is_oop(), "Oops, not an oop!"); | |
| 5758 assert(_mark_bit_map->isMarked((HeapWord*)obj_to_scan), "Stole an unmarked oop?"); | |
| 5759 // Do scanning work | |
| 5760 obj_to_scan->oop_iterate(keep_alive); | |
| 5761 // Loop around, finish this work, and try to steal some more | |
| 5762 } else if (terminator()->offer_termination()) { | |
| 5763 break; // nirvana from the infinite cycle | |
| 5764 } | |
| 5765 } | |
| 5766 NOT_PRODUCT( | |
| 5767 if (PrintCMSStatistics != 0) { | |
| 5768 gclog_or_tty->print("\n\t(%d: stole %d oops)", i, num_steals); | |
| 5769 } | |
| 5770 ) | |
| 5771 } | |
| 5772 | |
| 5773 void CMSRefProcTaskExecutor::execute(ProcessTask& task) | |
| 5774 { | |
| 5775 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 5776 WorkGang* workers = gch->workers(); | |
| 5777 assert(workers != NULL, "Need parallel worker threads."); | |
| 5778 int n_workers = workers->total_workers(); | |
| 5779 CMSRefProcTaskProxy rp_task(task, &_collector, | |
| 5780 _collector.ref_processor()->span(), | |
| 5781 _collector.markBitMap(), | |
| 5782 n_workers, _collector.task_queues()); | |
| 5783 workers->run_task(&rp_task); | |
| 5784 } | |
| 5785 | |
| 5786 void CMSRefProcTaskExecutor::execute(EnqueueTask& task) | |
| 5787 { | |
| 5788 | |
| 5789 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 5790 WorkGang* workers = gch->workers(); | |
| 5791 assert(workers != NULL, "Need parallel worker threads."); | |
| 5792 CMSRefEnqueueTaskProxy enq_task(task); | |
| 5793 workers->run_task(&enq_task); | |
| 5794 } | |
| 5795 | |
| 5796 void CMSCollector::refProcessingWork(bool asynch, bool clear_all_soft_refs) { | |
| 5797 | |
| 5798 ResourceMark rm; | |
| 5799 HandleMark hm; | |
| 5800 | |
| 5801 ReferenceProcessor* rp = ref_processor(); | |
| 5802 assert(rp->span().equals(_span), "Spans should be equal"); | |
|
453
c96030fff130
6684579: SoftReference processing can be made more efficient
ysr
parents:
452
diff
changeset
|
5803 assert(!rp->enqueuing_is_done(), "Enqueuing should not be complete"); |
|
c96030fff130
6684579: SoftReference processing can be made more efficient
ysr
parents:
452
diff
changeset
|
5804 // Process weak references. |
|
457
27a80744a83b
6778647: snap(), snap_policy() should be renamed setup(), setup_policy()
ysr
parents:
453
diff
changeset
|
5805 rp->setup_policy(clear_all_soft_refs); |
|
453
c96030fff130
6684579: SoftReference processing can be made more efficient
ysr
parents:
452
diff
changeset
|
5806 verify_work_stacks_empty(); |
|
c96030fff130
6684579: SoftReference processing can be made more efficient
ysr
parents:
452
diff
changeset
|
5807 |
| 0 | 5808 CMSKeepAliveClosure cmsKeepAliveClosure(this, _span, &_markBitMap, |
| 935 | 5809 &_markStack, &_revisitStack, |
| 5810 false /* !preclean */); | |
| 0 | 5811 CMSDrainMarkingStackClosure cmsDrainMarkingStackClosure(this, |
| 5812 _span, &_markBitMap, &_markStack, | |
|
452
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
5813 &cmsKeepAliveClosure, false /* !preclean */); |
| 0 | 5814 { |
| 5815 TraceTime t("weak refs processing", PrintGCDetails, false, gclog_or_tty); | |
| 5816 if (rp->processing_is_mt()) { | |
| 5817 CMSRefProcTaskExecutor task_executor(*this); | |
|
453
c96030fff130
6684579: SoftReference processing can be made more efficient
ysr
parents:
452
diff
changeset
|
5818 rp->process_discovered_references(&_is_alive_closure, |
| 0 | 5819 &cmsKeepAliveClosure, |
| 5820 &cmsDrainMarkingStackClosure, | |
| 5821 &task_executor); | |
| 5822 } else { | |
|
453
c96030fff130
6684579: SoftReference processing can be made more efficient
ysr
parents:
452
diff
changeset
|
5823 rp->process_discovered_references(&_is_alive_closure, |
| 0 | 5824 &cmsKeepAliveClosure, |
| 5825 &cmsDrainMarkingStackClosure, | |
| 5826 NULL); | |
| 5827 } | |
| 5828 verify_work_stacks_empty(); | |
| 5829 } | |
| 5830 | |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
5831 if (should_unload_classes()) { |
| 0 | 5832 { |
| 5833 TraceTime t("class unloading", PrintGCDetails, false, gclog_or_tty); | |
| 5834 | |
| 5835 // Follow SystemDictionary roots and unload classes | |
| 5836 bool purged_class = SystemDictionary::do_unloading(&_is_alive_closure); | |
| 5837 | |
| 5838 // Follow CodeCache roots and unload any methods marked for unloading | |
| 5839 CodeCache::do_unloading(&_is_alive_closure, | |
| 5840 &cmsKeepAliveClosure, | |
| 5841 purged_class); | |
| 5842 | |
| 5843 cmsDrainMarkingStackClosure.do_void(); | |
| 5844 verify_work_stacks_empty(); | |
| 5845 | |
| 5846 // Update subklass/sibling/implementor links in KlassKlass descendants | |
| 5847 assert(!_revisitStack.isEmpty(), "revisit stack should not be empty"); | |
| 5848 oop k; | |
| 5849 while ((k = _revisitStack.pop()) != NULL) { | |
| 5850 ((Klass*)(oopDesc*)k)->follow_weak_klass_links( | |
| 5851 &_is_alive_closure, | |
| 5852 &cmsKeepAliveClosure); | |
| 5853 } | |
| 5854 assert(!ClassUnloading || | |
| 5855 (_markStack.isEmpty() && overflow_list_is_empty()), | |
| 5856 "Should not have found new reachable objects"); | |
| 5857 assert(_revisitStack.isEmpty(), "revisit stack should have been drained"); | |
| 5858 cmsDrainMarkingStackClosure.do_void(); | |
| 5859 verify_work_stacks_empty(); | |
| 5860 } | |
| 5861 | |
| 5862 { | |
| 5863 TraceTime t("scrub symbol & string tables", PrintGCDetails, false, gclog_or_tty); | |
| 5864 // Now clean up stale oops in SymbolTable and StringTable | |
| 5865 SymbolTable::unlink(&_is_alive_closure); | |
| 5866 StringTable::unlink(&_is_alive_closure); | |
| 5867 } | |
| 5868 } | |
| 5869 | |
| 5870 verify_work_stacks_empty(); | |
| 5871 // Restore any preserved marks as a result of mark stack or | |
| 5872 // work queue overflow | |
| 5873 restore_preserved_marks_if_any(); // done single-threaded for now | |
| 5874 | |
| 5875 rp->set_enqueuing_is_done(true); | |
| 5876 if (rp->processing_is_mt()) { | |
| 5877 CMSRefProcTaskExecutor task_executor(*this); | |
| 5878 rp->enqueue_discovered_references(&task_executor); | |
| 5879 } else { | |
| 5880 rp->enqueue_discovered_references(NULL); | |
| 5881 } | |
| 5882 rp->verify_no_references_recorded(); | |
| 5883 assert(!rp->discovery_enabled(), "should have been disabled"); | |
| 5884 | |
| 5885 // JVMTI object tagging is based on JNI weak refs. If any of these | |
| 5886 // refs were cleared then JVMTI needs to update its maps and | |
| 5887 // maybe post ObjectFrees to agents. | |
| 5888 JvmtiExport::cms_ref_processing_epilogue(); | |
| 5889 } | |
| 5890 | |
| 5891 #ifndef PRODUCT | |
| 5892 void CMSCollector::check_correct_thread_executing() { | |
| 5893 Thread* t = Thread::current(); | |
| 5894 // Only the VM thread or the CMS thread should be here. | |
| 5895 assert(t->is_ConcurrentGC_thread() || t->is_VM_thread(), | |
| 5896 "Unexpected thread type"); | |
| 5897 // If this is the vm thread, the foreground process | |
| 5898 // should not be waiting. Note that _foregroundGCIsActive is | |
| 5899 // true while the foreground collector is waiting. | |
| 5900 if (_foregroundGCShouldWait) { | |
| 5901 // We cannot be the VM thread | |
| 5902 assert(t->is_ConcurrentGC_thread(), | |
| 5903 "Should be CMS thread"); | |
| 5904 } else { | |
| 5905 // We can be the CMS thread only if we are in a stop-world | |
| 5906 // phase of CMS collection. | |
| 5907 if (t->is_ConcurrentGC_thread()) { | |
| 5908 assert(_collectorState == InitialMarking || | |
| 5909 _collectorState == FinalMarking, | |
| 5910 "Should be a stop-world phase"); | |
| 5911 // The CMS thread should be holding the CMS_token. | |
| 5912 assert(ConcurrentMarkSweepThread::cms_thread_has_cms_token(), | |
| 5913 "Potential interference with concurrently " | |
| 5914 "executing VM thread"); | |
| 5915 } | |
| 5916 } | |
| 5917 } | |
| 5918 #endif | |
| 5919 | |
| 5920 void CMSCollector::sweep(bool asynch) { | |
| 5921 assert(_collectorState == Sweeping, "just checking"); | |
| 5922 check_correct_thread_executing(); | |
| 5923 verify_work_stacks_empty(); | |
| 5924 verify_overflow_empty(); | |
|
1145
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
5925 increment_sweep_count(); |
|
1703
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
5926 TraceCMSMemoryManagerStats tms(_collectorState); |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
5927 |
|
1145
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
5928 _inter_sweep_timer.stop(); |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
5929 _inter_sweep_estimate.sample(_inter_sweep_timer.seconds()); |
| 0 | 5930 size_policy()->avg_cms_free_at_sweep()->sample(_cmsGen->free()); |
| 5931 | |
| 5932 // PermGen verification support: If perm gen sweeping is disabled in | |
| 5933 // this cycle, we preserve the perm gen object "deadness" information | |
| 5934 // in the perm_gen_verify_bit_map. In order to do that we traverse | |
| 5935 // all blocks in perm gen and mark all dead objects. | |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
5936 if (verifying() && !should_unload_classes()) { |
| 0 | 5937 assert(perm_gen_verify_bit_map()->sizeInBits() != 0, |
| 5938 "Should have already been allocated"); | |
| 5939 MarkDeadObjectsClosure mdo(this, _permGen->cmsSpace(), | |
| 5940 markBitMap(), perm_gen_verify_bit_map()); | |
|
7
2faf283ce688
6621144: CMS: assertion failure "is_cms_thread == Thread::current()->is_ConcurrentGC_thread()"
ysr
parents:
0
diff
changeset
|
5941 if (asynch) { |
|
2faf283ce688
6621144: CMS: assertion failure "is_cms_thread == Thread::current()->is_ConcurrentGC_thread()"
ysr
parents:
0
diff
changeset
|
5942 CMSTokenSyncWithLocks ts(true, _permGen->freelistLock(), |
|
2faf283ce688
6621144: CMS: assertion failure "is_cms_thread == Thread::current()->is_ConcurrentGC_thread()"
ysr
parents:
0
diff
changeset
|
5943 bitMapLock()); |
|
2faf283ce688
6621144: CMS: assertion failure "is_cms_thread == Thread::current()->is_ConcurrentGC_thread()"
ysr
parents:
0
diff
changeset
|
5944 _permGen->cmsSpace()->blk_iterate(&mdo); |
|
2faf283ce688
6621144: CMS: assertion failure "is_cms_thread == Thread::current()->is_ConcurrentGC_thread()"
ysr
parents:
0
diff
changeset
|
5945 } else { |
|
2faf283ce688
6621144: CMS: assertion failure "is_cms_thread == Thread::current()->is_ConcurrentGC_thread()"
ysr
parents:
0
diff
changeset
|
5946 // In the case of synchronous sweep, we already have |
|
2faf283ce688
6621144: CMS: assertion failure "is_cms_thread == Thread::current()->is_ConcurrentGC_thread()"
ysr
parents:
0
diff
changeset
|
5947 // the requisite locks/tokens. |
|
2faf283ce688
6621144: CMS: assertion failure "is_cms_thread == Thread::current()->is_ConcurrentGC_thread()"
ysr
parents:
0
diff
changeset
|
5948 _permGen->cmsSpace()->blk_iterate(&mdo); |
|
2faf283ce688
6621144: CMS: assertion failure "is_cms_thread == Thread::current()->is_ConcurrentGC_thread()"
ysr
parents:
0
diff
changeset
|
5949 } |
| 0 | 5950 } |
| 5951 | |
|
1145
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
5952 assert(!_intra_sweep_timer.is_active(), "Should not be active"); |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
5953 _intra_sweep_timer.reset(); |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
5954 _intra_sweep_timer.start(); |
| 0 | 5955 if (asynch) { |
| 5956 TraceCPUTime tcpu(PrintGCDetails, true, gclog_or_tty); | |
| 5957 CMSPhaseAccounting pa(this, "sweep", !PrintGCDetails); | |
| 5958 // First sweep the old gen then the perm gen | |
| 5959 { | |
| 5960 CMSTokenSyncWithLocks ts(true, _cmsGen->freelistLock(), | |
| 5961 bitMapLock()); | |
| 5962 sweepWork(_cmsGen, asynch); | |
| 5963 } | |
| 5964 | |
| 5965 // Now repeat for perm gen | |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
5966 if (should_unload_classes()) { |
| 0 | 5967 CMSTokenSyncWithLocks ts(true, _permGen->freelistLock(), |
| 5968 bitMapLock()); | |
| 5969 sweepWork(_permGen, asynch); | |
| 5970 } | |
| 5971 | |
| 5972 // Update Universe::_heap_*_at_gc figures. | |
| 5973 // We need all the free list locks to make the abstract state | |
| 5974 // transition from Sweeping to Resetting. See detailed note | |
| 5975 // further below. | |
| 5976 { | |
| 5977 CMSTokenSyncWithLocks ts(true, _cmsGen->freelistLock(), | |
| 5978 _permGen->freelistLock()); | |
| 5979 // Update heap occupancy information which is used as | |
| 5980 // input to soft ref clearing policy at the next gc. | |
| 5981 Universe::update_heap_info_at_gc(); | |
| 5982 _collectorState = Resizing; | |
| 5983 } | |
| 5984 } else { | |
| 5985 // already have needed locks | |
| 5986 sweepWork(_cmsGen, asynch); | |
| 5987 | |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
5988 if (should_unload_classes()) { |
| 0 | 5989 sweepWork(_permGen, asynch); |
| 5990 } | |
| 5991 // Update heap occupancy information which is used as | |
| 5992 // input to soft ref clearing policy at the next gc. | |
| 5993 Universe::update_heap_info_at_gc(); | |
| 5994 _collectorState = Resizing; | |
| 5995 } | |
| 5996 verify_work_stacks_empty(); | |
| 5997 verify_overflow_empty(); | |
| 5998 | |
|
1145
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
5999 _intra_sweep_timer.stop(); |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
6000 _intra_sweep_estimate.sample(_intra_sweep_timer.seconds()); |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
6001 |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
6002 _inter_sweep_timer.reset(); |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
6003 _inter_sweep_timer.start(); |
| 0 | 6004 |
| 6005 update_time_of_last_gc(os::javaTimeMillis()); | |
| 6006 | |
| 6007 // NOTE on abstract state transitions: | |
| 6008 // Mutators allocate-live and/or mark the mod-union table dirty | |
| 6009 // based on the state of the collection. The former is done in | |
| 6010 // the interval [Marking, Sweeping] and the latter in the interval | |
| 6011 // [Marking, Sweeping). Thus the transitions into the Marking state | |
| 6012 // and out of the Sweeping state must be synchronously visible | |
| 6013 // globally to the mutators. | |
| 6014 // The transition into the Marking state happens with the world | |
| 6015 // stopped so the mutators will globally see it. Sweeping is | |
| 6016 // done asynchronously by the background collector so the transition | |
| 6017 // from the Sweeping state to the Resizing state must be done | |
| 6018 // under the freelistLock (as is the check for whether to | |
| 6019 // allocate-live and whether to dirty the mod-union table). | |
| 6020 assert(_collectorState == Resizing, "Change of collector state to" | |
| 6021 " Resizing must be done under the freelistLocks (plural)"); | |
| 6022 | |
| 6023 // Now that sweeping has been completed, if the GCH's | |
| 6024 // incremental_collection_will_fail flag is set, clear it, | |
| 6025 // thus inviting a younger gen collection to promote into | |
| 6026 // this generation. If such a promotion may still fail, | |
| 6027 // the flag will be set again when a young collection is | |
| 6028 // attempted. | |
| 6029 // I think the incremental_collection_will_fail flag's use | |
| 6030 // is specific to a 2 generation collection policy, so i'll | |
| 6031 // assert that that's the configuration we are operating within. | |
| 6032 // The use of the flag can and should be generalized appropriately | |
| 6033 // in the future to deal with a general n-generation system. | |
| 6034 | |
| 6035 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 6036 assert(gch->collector_policy()->is_two_generation_policy(), | |
| 6037 "Resetting of incremental_collection_will_fail flag" | |
| 6038 " may be incorrect otherwise"); | |
| 6039 gch->clear_incremental_collection_will_fail(); | |
| 6040 gch->update_full_collections_completed(_collection_count_start); | |
| 6041 } | |
| 6042 | |
| 6043 // FIX ME!!! Looks like this belongs in CFLSpace, with | |
| 6044 // CMSGen merely delegating to it. | |
| 6045 void ConcurrentMarkSweepGeneration::setNearLargestChunk() { | |
|
1145
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
6046 double nearLargestPercent = FLSLargestBlockCoalesceProximity; |
| 0 | 6047 HeapWord* minAddr = _cmsSpace->bottom(); |
| 6048 HeapWord* largestAddr = | |
| 6049 (HeapWord*) _cmsSpace->dictionary()->findLargestDict(); | |
|
1145
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
6050 if (largestAddr == NULL) { |
| 0 | 6051 // The dictionary appears to be empty. In this case |
| 6052 // try to coalesce at the end of the heap. | |
| 6053 largestAddr = _cmsSpace->end(); | |
| 6054 } | |
| 6055 size_t largestOffset = pointer_delta(largestAddr, minAddr); | |
| 6056 size_t nearLargestOffset = | |
| 6057 (size_t)((double)largestOffset * nearLargestPercent) - MinChunkSize; | |
|
1145
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
6058 if (PrintFLSStatistics != 0) { |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
6059 gclog_or_tty->print_cr( |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
6060 "CMS: Large Block: " PTR_FORMAT ";" |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
6061 " Proximity: " PTR_FORMAT " -> " PTR_FORMAT, |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
6062 largestAddr, |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
6063 _cmsSpace->nearLargestChunk(), minAddr + nearLargestOffset); |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
6064 } |
| 0 | 6065 _cmsSpace->set_nearLargestChunk(minAddr + nearLargestOffset); |
| 6066 } | |
| 6067 | |
| 6068 bool ConcurrentMarkSweepGeneration::isNearLargestChunk(HeapWord* addr) { | |
| 6069 return addr >= _cmsSpace->nearLargestChunk(); | |
| 6070 } | |
| 6071 | |
| 6072 FreeChunk* ConcurrentMarkSweepGeneration::find_chunk_at_end() { | |
| 6073 return _cmsSpace->find_chunk_at_end(); | |
| 6074 } | |
| 6075 | |
| 6076 void ConcurrentMarkSweepGeneration::update_gc_stats(int current_level, | |
| 6077 bool full) { | |
| 6078 // The next lower level has been collected. Gather any statistics | |
| 6079 // that are of interest at this point. | |
| 6080 if (!full && (current_level + 1) == level()) { | |
| 6081 // Gather statistics on the young generation collection. | |
| 6082 collector()->stats().record_gc0_end(used()); | |
| 6083 } | |
| 6084 } | |
| 6085 | |
| 6086 CMSAdaptiveSizePolicy* ConcurrentMarkSweepGeneration::size_policy() { | |
| 6087 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 6088 assert(gch->kind() == CollectedHeap::GenCollectedHeap, | |
| 6089 "Wrong type of heap"); | |
| 6090 CMSAdaptiveSizePolicy* sp = (CMSAdaptiveSizePolicy*) | |
| 6091 gch->gen_policy()->size_policy(); | |
| 6092 assert(sp->is_gc_cms_adaptive_size_policy(), | |
| 6093 "Wrong type of size policy"); | |
| 6094 return sp; | |
| 6095 } | |
| 6096 | |
| 6097 void ConcurrentMarkSweepGeneration::rotate_debug_collection_type() { | |
| 6098 if (PrintGCDetails && Verbose) { | |
| 6099 gclog_or_tty->print("Rotate from %d ", _debug_collection_type); | |
| 6100 } | |
| 6101 _debug_collection_type = (CollectionTypes) (_debug_collection_type + 1); | |
| 6102 _debug_collection_type = | |
| 6103 (CollectionTypes) (_debug_collection_type % Unknown_collection_type); | |
| 6104 if (PrintGCDetails && Verbose) { | |
| 6105 gclog_or_tty->print_cr("to %d ", _debug_collection_type); | |
| 6106 } | |
| 6107 } | |
| 6108 | |
| 6109 void CMSCollector::sweepWork(ConcurrentMarkSweepGeneration* gen, | |
| 6110 bool asynch) { | |
| 6111 // We iterate over the space(s) underlying this generation, | |
| 6112 // checking the mark bit map to see if the bits corresponding | |
| 6113 // to specific blocks are marked or not. Blocks that are | |
| 6114 // marked are live and are not swept up. All remaining blocks | |
| 6115 // are swept up, with coalescing on-the-fly as we sweep up | |
| 6116 // contiguous free and/or garbage blocks: | |
| 6117 // We need to ensure that the sweeper synchronizes with allocators | |
| 6118 // and stop-the-world collectors. In particular, the following | |
| 6119 // locks are used: | |
| 6120 // . CMS token: if this is held, a stop the world collection cannot occur | |
| 6121 // . freelistLock: if this is held no allocation can occur from this | |
| 6122 // generation by another thread | |
| 6123 // . bitMapLock: if this is held, no other thread can access or update | |
| 6124 // | |
| 6125 | |
| 6126 // Note that we need to hold the freelistLock if we use | |
| 6127 // block iterate below; else the iterator might go awry if | |
| 6128 // a mutator (or promotion) causes block contents to change | |
| 6129 // (for instance if the allocator divvies up a block). | |
| 6130 // If we hold the free list lock, for all practical purposes | |
| 6131 // young generation GC's can't occur (they'll usually need to | |
| 6132 // promote), so we might as well prevent all young generation | |
| 6133 // GC's while we do a sweeping step. For the same reason, we might | |
| 6134 // as well take the bit map lock for the entire duration | |
| 6135 | |
| 6136 // check that we hold the requisite locks | |
| 6137 assert(have_cms_token(), "Should hold cms token"); | |
| 6138 assert( (asynch && ConcurrentMarkSweepThread::cms_thread_has_cms_token()) | |
| 6139 || (!asynch && ConcurrentMarkSweepThread::vm_thread_has_cms_token()), | |
| 6140 "Should possess CMS token to sweep"); | |
| 6141 assert_lock_strong(gen->freelistLock()); | |
| 6142 assert_lock_strong(bitMapLock()); | |
| 6143 | |
|
1145
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
6144 assert(!_inter_sweep_timer.is_active(), "Was switched off in an outer context"); |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
6145 assert(_intra_sweep_timer.is_active(), "Was switched on in an outer context"); |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
6146 gen->cmsSpace()->beginSweepFLCensus((float)(_inter_sweep_timer.seconds()), |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
6147 _inter_sweep_estimate.padded_average(), |
|
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
6148 _intra_sweep_estimate.padded_average()); |
| 0 | 6149 gen->setNearLargestChunk(); |
| 6150 | |
| 6151 { | |
| 6152 SweepClosure sweepClosure(this, gen, &_markBitMap, | |
| 6153 CMSYield && asynch); | |
| 6154 gen->cmsSpace()->blk_iterate_careful(&sweepClosure); | |
| 6155 // We need to free-up/coalesce garbage/blocks from a | |
| 6156 // co-terminal free run. This is done in the SweepClosure | |
| 6157 // destructor; so, do not remove this scope, else the | |
| 6158 // end-of-sweep-census below will be off by a little bit. | |
| 6159 } | |
| 6160 gen->cmsSpace()->sweep_completed(); | |
|
1145
e018e6884bd8
6631166: CMS: better heuristics when combatting fragmentation
ysr
parents:
1085
diff
changeset
|
6161 gen->cmsSpace()->endSweepFLCensus(sweep_count()); |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
6162 if (should_unload_classes()) { // unloaded classes this cycle, |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
6163 _concurrent_cycles_since_last_unload = 0; // ... reset count |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
6164 } else { // did not unload classes, |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
6165 _concurrent_cycles_since_last_unload++; // ... increment count |
|
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
6166 } |
| 0 | 6167 } |
| 6168 | |
| 6169 // Reset CMS data structures (for now just the marking bit map) | |
| 6170 // preparatory for the next cycle. | |
| 6171 void CMSCollector::reset(bool asynch) { | |
| 6172 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 6173 CMSAdaptiveSizePolicy* sp = size_policy(); | |
| 6174 AdaptiveSizePolicyOutput(sp, gch->total_collections()); | |
| 6175 if (asynch) { | |
| 6176 CMSTokenSyncWithLocks ts(true, bitMapLock()); | |
| 6177 | |
| 6178 // If the state is not "Resetting", the foreground thread | |
| 6179 // has done a collection and the resetting. | |
| 6180 if (_collectorState != Resetting) { | |
| 6181 assert(_collectorState == Idling, "The state should only change" | |
| 6182 " because the foreground collector has finished the collection"); | |
| 6183 return; | |
| 6184 } | |
| 6185 | |
| 6186 // Clear the mark bitmap (no grey objects to start with) | |
| 6187 // for the next cycle. | |
| 6188 TraceCPUTime tcpu(PrintGCDetails, true, gclog_or_tty); | |
| 6189 CMSPhaseAccounting cmspa(this, "reset", !PrintGCDetails); | |
| 6190 | |
| 6191 HeapWord* curAddr = _markBitMap.startWord(); | |
| 6192 while (curAddr < _markBitMap.endWord()) { | |
| 6193 size_t remaining = pointer_delta(_markBitMap.endWord(), curAddr); | |
| 6194 MemRegion chunk(curAddr, MIN2(CMSBitMapYieldQuantum, remaining)); | |
| 6195 _markBitMap.clear_large_range(chunk); | |
| 6196 if (ConcurrentMarkSweepThread::should_yield() && | |
| 6197 !foregroundGCIsActive() && | |
| 6198 CMSYield) { | |
| 6199 assert(ConcurrentMarkSweepThread::cms_thread_has_cms_token(), | |
| 6200 "CMS thread should hold CMS token"); | |
| 6201 assert_lock_strong(bitMapLock()); | |
| 6202 bitMapLock()->unlock(); | |
| 6203 ConcurrentMarkSweepThread::desynchronize(true); | |
| 6204 ConcurrentMarkSweepThread::acknowledge_yield_request(); | |
| 6205 stopTimer(); | |
| 6206 if (PrintCMSStatistics != 0) { | |
| 6207 incrementYields(); | |
| 6208 } | |
| 6209 icms_wait(); | |
| 6210 | |
| 6211 // See the comment in coordinator_yield() | |
| 6212 for (unsigned i = 0; i < CMSYieldSleepCount && | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6213 ConcurrentMarkSweepThread::should_yield() && |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6214 !CMSCollector::foregroundGCIsActive(); ++i) { |
| 0 | 6215 os::sleep(Thread::current(), 1, false); |
| 6216 ConcurrentMarkSweepThread::acknowledge_yield_request(); | |
| 6217 } | |
| 6218 | |
| 6219 ConcurrentMarkSweepThread::synchronize(true); | |
| 6220 bitMapLock()->lock_without_safepoint_check(); | |
| 6221 startTimer(); | |
| 6222 } | |
| 6223 curAddr = chunk.end(); | |
| 6224 } | |
|
1387
0bfd3fb24150
6858496: Clear all SoftReferences before an out-of-memory due to GC overhead limit.
jmasa
parents:
1289
diff
changeset
|
6225 // A successful mostly concurrent collection has been done. |
|
0bfd3fb24150
6858496: Clear all SoftReferences before an out-of-memory due to GC overhead limit.
jmasa
parents:
1289
diff
changeset
|
6226 // Because only the full (i.e., concurrent mode failure) collections |
|
0bfd3fb24150
6858496: Clear all SoftReferences before an out-of-memory due to GC overhead limit.
jmasa
parents:
1289
diff
changeset
|
6227 // are being measured for gc overhead limits, clean the "near" flag |
|
0bfd3fb24150
6858496: Clear all SoftReferences before an out-of-memory due to GC overhead limit.
jmasa
parents:
1289
diff
changeset
|
6228 // and count. |
|
0bfd3fb24150
6858496: Clear all SoftReferences before an out-of-memory due to GC overhead limit.
jmasa
parents:
1289
diff
changeset
|
6229 sp->reset_gc_overhead_limit_count(); |
| 0 | 6230 _collectorState = Idling; |
| 6231 } else { | |
| 6232 // already have the lock | |
| 6233 assert(_collectorState == Resetting, "just checking"); | |
| 6234 assert_lock_strong(bitMapLock()); | |
| 6235 _markBitMap.clear_all(); | |
| 6236 _collectorState = Idling; | |
| 6237 } | |
| 6238 | |
| 6239 // Stop incremental mode after a cycle completes, so that any future cycles | |
| 6240 // are triggered by allocation. | |
| 6241 stop_icms(); | |
| 6242 | |
| 6243 NOT_PRODUCT( | |
| 6244 if (RotateCMSCollectionTypes) { | |
| 6245 _cmsGen->rotate_debug_collection_type(); | |
| 6246 } | |
| 6247 ) | |
| 6248 } | |
| 6249 | |
| 6250 void CMSCollector::do_CMS_operation(CMS_op_type op) { | |
| 6251 gclog_or_tty->date_stamp(PrintGC && PrintGCDateStamps); | |
| 6252 TraceCPUTime tcpu(PrintGCDetails, true, gclog_or_tty); | |
| 6253 TraceTime t("GC", PrintGC, !PrintGCDetails, gclog_or_tty); | |
| 6254 TraceCollectorStats tcs(counters()); | |
| 6255 | |
| 6256 switch (op) { | |
| 6257 case CMS_op_checkpointRootsInitial: { | |
| 6258 checkpointRootsInitial(true); // asynch | |
| 6259 if (PrintGC) { | |
| 6260 _cmsGen->printOccupancy("initial-mark"); | |
| 6261 } | |
| 6262 break; | |
| 6263 } | |
| 6264 case CMS_op_checkpointRootsFinal: { | |
| 6265 checkpointRootsFinal(true, // asynch | |
| 6266 false, // !clear_all_soft_refs | |
| 6267 false); // !init_mark_was_synchronous | |
| 6268 if (PrintGC) { | |
| 6269 _cmsGen->printOccupancy("remark"); | |
| 6270 } | |
| 6271 break; | |
| 6272 } | |
| 6273 default: | |
| 6274 fatal("No such CMS_op"); | |
| 6275 } | |
| 6276 } | |
| 6277 | |
| 6278 #ifndef PRODUCT | |
| 6279 size_t const CMSCollector::skip_header_HeapWords() { | |
| 6280 return FreeChunk::header_size(); | |
| 6281 } | |
| 6282 | |
| 6283 // Try and collect here conditions that should hold when | |
| 6284 // CMS thread is exiting. The idea is that the foreground GC | |
| 6285 // thread should not be blocked if it wants to terminate | |
| 6286 // the CMS thread and yet continue to run the VM for a while | |
| 6287 // after that. | |
| 6288 void CMSCollector::verify_ok_to_terminate() const { | |
| 6289 assert(Thread::current()->is_ConcurrentGC_thread(), | |
| 6290 "should be called by CMS thread"); | |
| 6291 assert(!_foregroundGCShouldWait, "should be false"); | |
| 6292 // We could check here that all the various low-level locks | |
| 6293 // are not held by the CMS thread, but that is overkill; see | |
| 6294 // also CMSThread::verify_ok_to_terminate() where the CGC_lock | |
| 6295 // is checked. | |
| 6296 } | |
| 6297 #endif | |
| 6298 | |
| 6299 size_t CMSCollector::block_size_using_printezis_bits(HeapWord* addr) const { | |
|
453
c96030fff130
6684579: SoftReference processing can be made more efficient
ysr
parents:
452
diff
changeset
|
6300 assert(_markBitMap.isMarked(addr) && _markBitMap.isMarked(addr + 1), |
|
c96030fff130
6684579: SoftReference processing can be made more efficient
ysr
parents:
452
diff
changeset
|
6301 "missing Printezis mark?"); |
| 0 | 6302 HeapWord* nextOneAddr = _markBitMap.getNextMarkedWordAddress(addr + 2); |
| 6303 size_t size = pointer_delta(nextOneAddr + 1, addr); | |
| 6304 assert(size == CompactibleFreeListSpace::adjustObjectSize(size), | |
| 6305 "alignment problem"); | |
| 6306 assert(size >= 3, "Necessary for Printezis marks to work"); | |
| 6307 return size; | |
| 6308 } | |
| 6309 | |
| 6310 // A variant of the above (block_size_using_printezis_bits()) except | |
| 6311 // that we return 0 if the P-bits are not yet set. | |
| 6312 size_t CMSCollector::block_size_if_printezis_bits(HeapWord* addr) const { | |
| 6313 if (_markBitMap.isMarked(addr)) { | |
| 6314 assert(_markBitMap.isMarked(addr + 1), "Missing Printezis bit?"); | |
| 6315 HeapWord* nextOneAddr = _markBitMap.getNextMarkedWordAddress(addr + 2); | |
| 6316 size_t size = pointer_delta(nextOneAddr + 1, addr); | |
| 6317 assert(size == CompactibleFreeListSpace::adjustObjectSize(size), | |
| 6318 "alignment problem"); | |
| 6319 assert(size >= 3, "Necessary for Printezis marks to work"); | |
| 6320 return size; | |
| 6321 } else { | |
| 6322 assert(!_markBitMap.isMarked(addr + 1), "Bit map inconsistency?"); | |
| 6323 return 0; | |
| 6324 } | |
| 6325 } | |
| 6326 | |
| 6327 HeapWord* CMSCollector::next_card_start_after_block(HeapWord* addr) const { | |
| 6328 size_t sz = 0; | |
| 6329 oop p = (oop)addr; | |
| 187 | 6330 if (p->klass_or_null() != NULL && p->is_parsable()) { |
| 0 | 6331 sz = CompactibleFreeListSpace::adjustObjectSize(p->size()); |
| 6332 } else { | |
| 6333 sz = block_size_using_printezis_bits(addr); | |
| 6334 } | |
| 6335 assert(sz > 0, "size must be nonzero"); | |
| 6336 HeapWord* next_block = addr + sz; | |
| 6337 HeapWord* next_card = (HeapWord*)round_to((uintptr_t)next_block, | |
| 6338 CardTableModRefBS::card_size); | |
| 6339 assert(round_down((uintptr_t)addr, CardTableModRefBS::card_size) < | |
| 6340 round_down((uintptr_t)next_card, CardTableModRefBS::card_size), | |
| 6341 "must be different cards"); | |
| 6342 return next_card; | |
| 6343 } | |
| 6344 | |
| 6345 | |
| 6346 // CMS Bit Map Wrapper ///////////////////////////////////////// | |
| 6347 | |
| 6348 // Construct a CMS bit map infrastructure, but don't create the | |
| 6349 // bit vector itself. That is done by a separate call CMSBitMap::allocate() | |
| 6350 // further below. | |
| 6351 CMSBitMap::CMSBitMap(int shifter, int mutex_rank, const char* mutex_name): | |
|
342
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
6352 _bm(), |
| 0 | 6353 _shifter(shifter), |
| 6354 _lock(mutex_rank >= 0 ? new Mutex(mutex_rank, mutex_name, true) : NULL) | |
| 6355 { | |
| 6356 _bmStartWord = 0; | |
| 6357 _bmWordSize = 0; | |
| 6358 } | |
| 6359 | |
| 6360 bool CMSBitMap::allocate(MemRegion mr) { | |
| 6361 _bmStartWord = mr.start(); | |
| 6362 _bmWordSize = mr.word_size(); | |
| 6363 ReservedSpace brs(ReservedSpace::allocation_align_size_up( | |
| 6364 (_bmWordSize >> (_shifter + LogBitsPerByte)) + 1)); | |
| 6365 if (!brs.is_reserved()) { | |
| 6366 warning("CMS bit map allocation failure"); | |
| 6367 return false; | |
| 6368 } | |
| 6369 // For now we'll just commit all of the bit map up fromt. | |
| 6370 // Later on we'll try to be more parsimonious with swap. | |
| 6371 if (!_virtual_space.initialize(brs, brs.size())) { | |
| 6372 warning("CMS bit map backing store failure"); | |
| 6373 return false; | |
| 6374 } | |
| 6375 assert(_virtual_space.committed_size() == brs.size(), | |
| 6376 "didn't reserve backing store for all of CMS bit map?"); | |
|
342
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
6377 _bm.set_map((BitMap::bm_word_t*)_virtual_space.low()); |
| 0 | 6378 assert(_virtual_space.committed_size() << (_shifter + LogBitsPerByte) >= |
| 6379 _bmWordSize, "inconsistency in bit map sizing"); | |
| 6380 _bm.set_size(_bmWordSize >> _shifter); | |
| 6381 | |
| 6382 // bm.clear(); // can we rely on getting zero'd memory? verify below | |
| 6383 assert(isAllClear(), | |
| 6384 "Expected zero'd memory from ReservedSpace constructor"); | |
| 6385 assert(_bm.size() == heapWordDiffToOffsetDiff(sizeInWords()), | |
| 6386 "consistency check"); | |
| 6387 return true; | |
| 6388 } | |
| 6389 | |
| 6390 void CMSBitMap::dirty_range_iterate_clear(MemRegion mr, MemRegionClosure* cl) { | |
| 6391 HeapWord *next_addr, *end_addr, *last_addr; | |
| 6392 assert_locked(); | |
| 6393 assert(covers(mr), "out-of-range error"); | |
| 6394 // XXX assert that start and end are appropriately aligned | |
| 6395 for (next_addr = mr.start(), end_addr = mr.end(); | |
| 6396 next_addr < end_addr; next_addr = last_addr) { | |
| 6397 MemRegion dirty_region = getAndClearMarkedRegion(next_addr, end_addr); | |
| 6398 last_addr = dirty_region.end(); | |
| 6399 if (!dirty_region.is_empty()) { | |
| 6400 cl->do_MemRegion(dirty_region); | |
| 6401 } else { | |
| 6402 assert(last_addr == end_addr, "program logic"); | |
| 6403 return; | |
| 6404 } | |
| 6405 } | |
| 6406 } | |
| 6407 | |
| 6408 #ifndef PRODUCT | |
| 6409 void CMSBitMap::assert_locked() const { | |
| 6410 CMSLockVerifier::assert_locked(lock()); | |
| 6411 } | |
| 6412 | |
| 6413 bool CMSBitMap::covers(MemRegion mr) const { | |
| 6414 // assert(_bm.map() == _virtual_space.low(), "map inconsistency"); | |
| 6415 assert((size_t)_bm.size() == (_bmWordSize >> _shifter), | |
| 6416 "size inconsistency"); | |
| 6417 return (mr.start() >= _bmStartWord) && | |
| 6418 (mr.end() <= endWord()); | |
| 6419 } | |
| 6420 | |
| 6421 bool CMSBitMap::covers(HeapWord* start, size_t size) const { | |
| 6422 return (start >= _bmStartWord && (start + size) <= endWord()); | |
| 6423 } | |
| 6424 | |
| 6425 void CMSBitMap::verifyNoOneBitsInRange(HeapWord* left, HeapWord* right) { | |
| 6426 // verify that there are no 1 bits in the interval [left, right) | |
| 6427 FalseBitMapClosure falseBitMapClosure; | |
| 6428 iterate(&falseBitMapClosure, left, right); | |
| 6429 } | |
| 6430 | |
| 6431 void CMSBitMap::region_invariant(MemRegion mr) | |
| 6432 { | |
| 6433 assert_locked(); | |
| 6434 // mr = mr.intersection(MemRegion(_bmStartWord, _bmWordSize)); | |
| 6435 assert(!mr.is_empty(), "unexpected empty region"); | |
| 6436 assert(covers(mr), "mr should be covered by bit map"); | |
| 6437 // convert address range into offset range | |
| 6438 size_t start_ofs = heapWordToOffset(mr.start()); | |
| 6439 // Make sure that end() is appropriately aligned | |
| 6440 assert(mr.end() == (HeapWord*)round_to((intptr_t)mr.end(), | |
| 6441 (1 << (_shifter+LogHeapWordSize))), | |
| 6442 "Misaligned mr.end()"); | |
| 6443 size_t end_ofs = heapWordToOffset(mr.end()); | |
| 6444 assert(end_ofs > start_ofs, "Should mark at least one bit"); | |
| 6445 } | |
| 6446 | |
| 6447 #endif | |
| 6448 | |
| 6449 bool CMSMarkStack::allocate(size_t size) { | |
| 6450 // allocate a stack of the requisite depth | |
| 6451 ReservedSpace rs(ReservedSpace::allocation_align_size_up( | |
| 6452 size * sizeof(oop))); | |
| 6453 if (!rs.is_reserved()) { | |
| 6454 warning("CMSMarkStack allocation failure"); | |
| 6455 return false; | |
| 6456 } | |
| 6457 if (!_virtual_space.initialize(rs, rs.size())) { | |
| 6458 warning("CMSMarkStack backing store failure"); | |
| 6459 return false; | |
| 6460 } | |
| 6461 assert(_virtual_space.committed_size() == rs.size(), | |
| 6462 "didn't reserve backing store for all of CMS stack?"); | |
| 6463 _base = (oop*)(_virtual_space.low()); | |
| 6464 _index = 0; | |
| 6465 _capacity = size; | |
| 6466 NOT_PRODUCT(_max_depth = 0); | |
| 6467 return true; | |
| 6468 } | |
| 6469 | |
| 6470 // XXX FIX ME !!! In the MT case we come in here holding a | |
| 6471 // leaf lock. For printing we need to take a further lock | |
| 6472 // which has lower rank. We need to recallibrate the two | |
| 6473 // lock-ranks involved in order to be able to rpint the | |
| 6474 // messages below. (Or defer the printing to the caller. | |
| 6475 // For now we take the expedient path of just disabling the | |
| 6476 // messages for the problematic case.) | |
| 6477 void CMSMarkStack::expand() { | |
| 1284 | 6478 assert(_capacity <= MarkStackSizeMax, "stack bigger than permitted"); |
| 6479 if (_capacity == MarkStackSizeMax) { | |
| 0 | 6480 if (_hit_limit++ == 0 && !CMSConcurrentMTEnabled && PrintGCDetails) { |
| 6481 // We print a warning message only once per CMS cycle. | |
| 6482 gclog_or_tty->print_cr(" (benign) Hit CMSMarkStack max size limit"); | |
| 6483 } | |
| 6484 return; | |
| 6485 } | |
| 6486 // Double capacity if possible | |
| 1284 | 6487 size_t new_capacity = MIN2(_capacity*2, MarkStackSizeMax); |
| 0 | 6488 // Do not give up existing stack until we have managed to |
| 6489 // get the double capacity that we desired. | |
| 6490 ReservedSpace rs(ReservedSpace::allocation_align_size_up( | |
| 6491 new_capacity * sizeof(oop))); | |
| 6492 if (rs.is_reserved()) { | |
| 6493 // Release the backing store associated with old stack | |
| 6494 _virtual_space.release(); | |
| 6495 // Reinitialize virtual space for new stack | |
| 6496 if (!_virtual_space.initialize(rs, rs.size())) { | |
| 6497 fatal("Not enough swap for expanded marking stack"); | |
| 6498 } | |
| 6499 _base = (oop*)(_virtual_space.low()); | |
| 6500 _index = 0; | |
| 6501 _capacity = new_capacity; | |
| 6502 } else if (_failed_double++ == 0 && !CMSConcurrentMTEnabled && PrintGCDetails) { | |
| 6503 // Failed to double capacity, continue; | |
| 6504 // we print a detail message only once per CMS cycle. | |
| 6505 gclog_or_tty->print(" (benign) Failed to expand marking stack from "SIZE_FORMAT"K to " | |
| 6506 SIZE_FORMAT"K", | |
| 6507 _capacity / K, new_capacity / K); | |
| 6508 } | |
| 6509 } | |
| 6510 | |
| 6511 | |
| 6512 // Closures | |
| 6513 // XXX: there seems to be a lot of code duplication here; | |
| 6514 // should refactor and consolidate common code. | |
| 6515 | |
| 6516 // This closure is used to mark refs into the CMS generation in | |
| 6517 // the CMS bit map. Called at the first checkpoint. This closure | |
| 6518 // assumes that we do not need to re-mark dirty cards; if the CMS | |
| 6519 // generation on which this is used is not an oldest (modulo perm gen) | |
| 6520 // generation then this will lose younger_gen cards! | |
| 6521 | |
| 6522 MarkRefsIntoClosure::MarkRefsIntoClosure( | |
|
994
753cf9794df9
6885169: merge of 4957990 and 6863023 causes conflict on do_nmethods
jrose
parents:
993
diff
changeset
|
6523 MemRegion span, CMSBitMap* bitMap): |
| 0 | 6524 _span(span), |
|
994
753cf9794df9
6885169: merge of 4957990 and 6863023 causes conflict on do_nmethods
jrose
parents:
993
diff
changeset
|
6525 _bitMap(bitMap) |
| 0 | 6526 { |
| 6527 assert(_ref_processor == NULL, "deliberately left NULL"); | |
| 6528 assert(_bitMap->covers(_span), "_bitMap/_span mismatch"); | |
| 6529 } | |
| 6530 | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6531 void MarkRefsIntoClosure::do_oop(oop obj) { |
| 0 | 6532 // if p points into _span, then mark corresponding bit in _markBitMap |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6533 assert(obj->is_oop(), "expected an oop"); |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6534 HeapWord* addr = (HeapWord*)obj; |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6535 if (_span.contains(addr)) { |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6536 // this should be made more efficient |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6537 _bitMap->mark(addr); |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6538 } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6539 } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6540 |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6541 void MarkRefsIntoClosure::do_oop(oop* p) { MarkRefsIntoClosure::do_oop_work(p); } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6542 void MarkRefsIntoClosure::do_oop(narrowOop* p) { MarkRefsIntoClosure::do_oop_work(p); } |
| 0 | 6543 |
| 6544 // A variant of the above, used for CMS marking verification. | |
| 6545 MarkRefsIntoVerifyClosure::MarkRefsIntoVerifyClosure( | |
|
994
753cf9794df9
6885169: merge of 4957990 and 6863023 causes conflict on do_nmethods
jrose
parents:
993
diff
changeset
|
6546 MemRegion span, CMSBitMap* verification_bm, CMSBitMap* cms_bm): |
| 0 | 6547 _span(span), |
| 6548 _verification_bm(verification_bm), | |
|
994
753cf9794df9
6885169: merge of 4957990 and 6863023 causes conflict on do_nmethods
jrose
parents:
993
diff
changeset
|
6549 _cms_bm(cms_bm) |
|
753cf9794df9
6885169: merge of 4957990 and 6863023 causes conflict on do_nmethods
jrose
parents:
993
diff
changeset
|
6550 { |
| 0 | 6551 assert(_ref_processor == NULL, "deliberately left NULL"); |
| 6552 assert(_verification_bm->covers(_span), "_verification_bm/_span mismatch"); | |
| 6553 } | |
| 6554 | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6555 void MarkRefsIntoVerifyClosure::do_oop(oop obj) { |
| 0 | 6556 // if p points into _span, then mark corresponding bit in _markBitMap |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6557 assert(obj->is_oop(), "expected an oop"); |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6558 HeapWord* addr = (HeapWord*)obj; |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6559 if (_span.contains(addr)) { |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6560 _verification_bm->mark(addr); |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6561 if (!_cms_bm->isMarked(addr)) { |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6562 oop(addr)->print(); |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6563 gclog_or_tty->print_cr(" (" INTPTR_FORMAT " should have been marked)", addr); |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6564 fatal("... aborting"); |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6565 } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6566 } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6567 } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6568 |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6569 void MarkRefsIntoVerifyClosure::do_oop(oop* p) { MarkRefsIntoVerifyClosure::do_oop_work(p); } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6570 void MarkRefsIntoVerifyClosure::do_oop(narrowOop* p) { MarkRefsIntoVerifyClosure::do_oop_work(p); } |
| 0 | 6571 |
| 6572 ////////////////////////////////////////////////// | |
| 6573 // MarkRefsIntoAndScanClosure | |
| 6574 ////////////////////////////////////////////////// | |
| 6575 | |
| 6576 MarkRefsIntoAndScanClosure::MarkRefsIntoAndScanClosure(MemRegion span, | |
| 6577 ReferenceProcessor* rp, | |
| 6578 CMSBitMap* bit_map, | |
| 6579 CMSBitMap* mod_union_table, | |
| 6580 CMSMarkStack* mark_stack, | |
| 6581 CMSMarkStack* revisit_stack, | |
| 6582 CMSCollector* collector, | |
| 6583 bool should_yield, | |
| 6584 bool concurrent_precleaning): | |
| 6585 _collector(collector), | |
| 6586 _span(span), | |
| 6587 _bit_map(bit_map), | |
| 6588 _mark_stack(mark_stack), | |
| 6589 _pushAndMarkClosure(collector, span, rp, bit_map, mod_union_table, | |
| 6590 mark_stack, revisit_stack, concurrent_precleaning), | |
| 6591 _yield(should_yield), | |
| 6592 _concurrent_precleaning(concurrent_precleaning), | |
| 6593 _freelistLock(NULL) | |
| 6594 { | |
| 6595 _ref_processor = rp; | |
| 6596 assert(_ref_processor != NULL, "_ref_processor shouldn't be NULL"); | |
| 6597 } | |
| 6598 | |
| 6599 // This closure is used to mark refs into the CMS generation at the | |
| 6600 // second (final) checkpoint, and to scan and transitively follow | |
| 6601 // the unmarked oops. It is also used during the concurrent precleaning | |
| 6602 // phase while scanning objects on dirty cards in the CMS generation. | |
| 6603 // The marks are made in the marking bit map and the marking stack is | |
| 6604 // used for keeping the (newly) grey objects during the scan. | |
| 6605 // The parallel version (Par_...) appears further below. | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6606 void MarkRefsIntoAndScanClosure::do_oop(oop obj) { |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6607 if (obj != NULL) { |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6608 assert(obj->is_oop(), "expected an oop"); |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6609 HeapWord* addr = (HeapWord*)obj; |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6610 assert(_mark_stack->isEmpty(), "pre-condition (eager drainage)"); |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6611 assert(_collector->overflow_list_is_empty(), |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6612 "overflow list should be empty"); |
| 0 | 6613 if (_span.contains(addr) && |
| 6614 !_bit_map->isMarked(addr)) { | |
| 6615 // mark bit map (object is now grey) | |
| 6616 _bit_map->mark(addr); | |
| 6617 // push on marking stack (stack should be empty), and drain the | |
| 6618 // stack by applying this closure to the oops in the oops popped | |
| 6619 // from the stack (i.e. blacken the grey objects) | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6620 bool res = _mark_stack->push(obj); |
| 0 | 6621 assert(res, "Should have space to push on empty stack"); |
| 6622 do { | |
| 6623 oop new_oop = _mark_stack->pop(); | |
| 6624 assert(new_oop != NULL && new_oop->is_oop(), "Expected an oop"); | |
| 6625 assert(new_oop->is_parsable(), "Found unparsable oop"); | |
| 6626 assert(_bit_map->isMarked((HeapWord*)new_oop), | |
| 6627 "only grey objects on this stack"); | |
| 6628 // iterate over the oops in this oop, marking and pushing | |
| 6629 // the ones in CMS heap (i.e. in _span). | |
| 6630 new_oop->oop_iterate(&_pushAndMarkClosure); | |
| 6631 // check if it's time to yield | |
| 6632 do_yield_check(); | |
| 6633 } while (!_mark_stack->isEmpty() || | |
| 6634 (!_concurrent_precleaning && take_from_overflow_list())); | |
| 6635 // if marking stack is empty, and we are not doing this | |
| 6636 // during precleaning, then check the overflow list | |
| 6637 } | |
| 6638 assert(_mark_stack->isEmpty(), "post-condition (eager drainage)"); | |
| 6639 assert(_collector->overflow_list_is_empty(), | |
| 6640 "overflow list was drained above"); | |
| 6641 // We could restore evacuated mark words, if any, used for | |
| 6642 // overflow list links here because the overflow list is | |
| 6643 // provably empty here. That would reduce the maximum | |
| 6644 // size requirements for preserved_{oop,mark}_stack. | |
| 6645 // But we'll just postpone it until we are all done | |
| 6646 // so we can just stream through. | |
| 6647 if (!_concurrent_precleaning && CMSOverflowEarlyRestoration) { | |
| 6648 _collector->restore_preserved_marks_if_any(); | |
| 6649 assert(_collector->no_preserved_marks(), "No preserved marks"); | |
| 6650 } | |
| 6651 assert(!CMSOverflowEarlyRestoration || _collector->no_preserved_marks(), | |
| 6652 "All preserved marks should have been restored above"); | |
| 6653 } | |
| 6654 } | |
| 6655 | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6656 void MarkRefsIntoAndScanClosure::do_oop(oop* p) { MarkRefsIntoAndScanClosure::do_oop_work(p); } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6657 void MarkRefsIntoAndScanClosure::do_oop(narrowOop* p) { MarkRefsIntoAndScanClosure::do_oop_work(p); } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6658 |
| 0 | 6659 void MarkRefsIntoAndScanClosure::do_yield_work() { |
| 6660 assert(ConcurrentMarkSweepThread::cms_thread_has_cms_token(), | |
| 6661 "CMS thread should hold CMS token"); | |
| 6662 assert_lock_strong(_freelistLock); | |
| 6663 assert_lock_strong(_bit_map->lock()); | |
| 6664 // relinquish the free_list_lock and bitMaplock() | |
| 935 | 6665 DEBUG_ONLY(RememberKlassesChecker mux(false);) |
| 0 | 6666 _bit_map->lock()->unlock(); |
| 6667 _freelistLock->unlock(); | |
| 6668 ConcurrentMarkSweepThread::desynchronize(true); | |
| 6669 ConcurrentMarkSweepThread::acknowledge_yield_request(); | |
| 6670 _collector->stopTimer(); | |
| 6671 GCPauseTimer p(_collector->size_policy()->concurrent_timer_ptr()); | |
| 6672 if (PrintCMSStatistics != 0) { | |
| 6673 _collector->incrementYields(); | |
| 6674 } | |
| 6675 _collector->icms_wait(); | |
| 6676 | |
| 6677 // See the comment in coordinator_yield() | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6678 for (unsigned i = 0; |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6679 i < CMSYieldSleepCount && |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6680 ConcurrentMarkSweepThread::should_yield() && |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6681 !CMSCollector::foregroundGCIsActive(); |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6682 ++i) { |
| 0 | 6683 os::sleep(Thread::current(), 1, false); |
| 6684 ConcurrentMarkSweepThread::acknowledge_yield_request(); | |
| 6685 } | |
| 6686 | |
| 6687 ConcurrentMarkSweepThread::synchronize(true); | |
| 6688 _freelistLock->lock_without_safepoint_check(); | |
| 6689 _bit_map->lock()->lock_without_safepoint_check(); | |
| 6690 _collector->startTimer(); | |
| 6691 } | |
| 6692 | |
| 6693 /////////////////////////////////////////////////////////// | |
| 6694 // Par_MarkRefsIntoAndScanClosure: a parallel version of | |
| 6695 // MarkRefsIntoAndScanClosure | |
| 6696 /////////////////////////////////////////////////////////// | |
| 6697 Par_MarkRefsIntoAndScanClosure::Par_MarkRefsIntoAndScanClosure( | |
| 6698 CMSCollector* collector, MemRegion span, ReferenceProcessor* rp, | |
| 6699 CMSBitMap* bit_map, OopTaskQueue* work_queue, CMSMarkStack* revisit_stack): | |
| 6700 _span(span), | |
| 6701 _bit_map(bit_map), | |
| 6702 _work_queue(work_queue), | |
| 6703 _low_water_mark(MIN2((uint)(work_queue->max_elems()/4), | |
| 6704 (uint)(CMSWorkQueueDrainThreshold * ParallelGCThreads))), | |
| 6705 _par_pushAndMarkClosure(collector, span, rp, bit_map, work_queue, | |
| 6706 revisit_stack) | |
| 6707 { | |
| 6708 _ref_processor = rp; | |
| 6709 assert(_ref_processor != NULL, "_ref_processor shouldn't be NULL"); | |
| 6710 } | |
| 6711 | |
| 6712 // This closure is used to mark refs into the CMS generation at the | |
| 6713 // second (final) checkpoint, and to scan and transitively follow | |
| 6714 // the unmarked oops. The marks are made in the marking bit map and | |
| 6715 // the work_queue is used for keeping the (newly) grey objects during | |
| 6716 // the scan phase whence they are also available for stealing by parallel | |
| 6717 // threads. Since the marking bit map is shared, updates are | |
| 6718 // synchronized (via CAS). | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6719 void Par_MarkRefsIntoAndScanClosure::do_oop(oop obj) { |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6720 if (obj != NULL) { |
| 0 | 6721 // Ignore mark word because this could be an already marked oop |
| 6722 // that may be chained at the end of the overflow list. | |
|
340
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
6723 assert(obj->is_oop(true), "expected an oop"); |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6724 HeapWord* addr = (HeapWord*)obj; |
| 0 | 6725 if (_span.contains(addr) && |
| 6726 !_bit_map->isMarked(addr)) { | |
| 6727 // mark bit map (object will become grey): | |
| 6728 // It is possible for several threads to be | |
| 6729 // trying to "claim" this object concurrently; | |
| 6730 // the unique thread that succeeds in marking the | |
| 6731 // object first will do the subsequent push on | |
| 6732 // to the work queue (or overflow list). | |
| 6733 if (_bit_map->par_mark(addr)) { | |
| 6734 // push on work_queue (which may not be empty), and trim the | |
| 6735 // queue to an appropriate length by applying this closure to | |
| 6736 // the oops in the oops popped from the stack (i.e. blacken the | |
| 6737 // grey objects) | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6738 bool res = _work_queue->push(obj); |
| 0 | 6739 assert(res, "Low water mark should be less than capacity?"); |
| 6740 trim_queue(_low_water_mark); | |
| 6741 } // Else, another thread claimed the object | |
| 6742 } | |
| 6743 } | |
| 6744 } | |
| 6745 | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6746 void Par_MarkRefsIntoAndScanClosure::do_oop(oop* p) { Par_MarkRefsIntoAndScanClosure::do_oop_work(p); } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6747 void Par_MarkRefsIntoAndScanClosure::do_oop(narrowOop* p) { Par_MarkRefsIntoAndScanClosure::do_oop_work(p); } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6748 |
| 0 | 6749 // This closure is used to rescan the marked objects on the dirty cards |
| 6750 // in the mod union table and the card table proper. | |
| 6751 size_t ScanMarkedObjectsAgainCarefullyClosure::do_object_careful_m( | |
| 6752 oop p, MemRegion mr) { | |
| 6753 | |
| 6754 size_t size = 0; | |
| 6755 HeapWord* addr = (HeapWord*)p; | |
| 6756 DEBUG_ONLY(_collector->verify_work_stacks_empty();) | |
| 6757 assert(_span.contains(addr), "we are scanning the CMS generation"); | |
| 6758 // check if it's time to yield | |
| 6759 if (do_yield_check()) { | |
| 6760 // We yielded for some foreground stop-world work, | |
| 6761 // and we have been asked to abort this ongoing preclean cycle. | |
| 6762 return 0; | |
| 6763 } | |
| 6764 if (_bitMap->isMarked(addr)) { | |
| 6765 // it's marked; is it potentially uninitialized? | |
| 187 | 6766 if (p->klass_or_null() != NULL) { |
|
518
0af8b0718fc9
6692899: CMS: many vm.parallel_class_loading tests fail with assert "missing Printezis mark"
jmasa
parents:
517
diff
changeset
|
6767 // If is_conc_safe is false, the object may be undergoing |
|
0af8b0718fc9
6692899: CMS: many vm.parallel_class_loading tests fail with assert "missing Printezis mark"
jmasa
parents:
517
diff
changeset
|
6768 // change by the VM outside a safepoint. Don't try to |
|
0af8b0718fc9
6692899: CMS: many vm.parallel_class_loading tests fail with assert "missing Printezis mark"
jmasa
parents:
517
diff
changeset
|
6769 // scan it, but rather leave it for the remark phase. |
|
0af8b0718fc9
6692899: CMS: many vm.parallel_class_loading tests fail with assert "missing Printezis mark"
jmasa
parents:
517
diff
changeset
|
6770 if (CMSPermGenPrecleaningEnabled && |
|
0af8b0718fc9
6692899: CMS: many vm.parallel_class_loading tests fail with assert "missing Printezis mark"
jmasa
parents:
517
diff
changeset
|
6771 (!p->is_conc_safe() || !p->is_parsable())) { |
| 0 | 6772 // Signal precleaning to redirty the card since |
| 6773 // the klass pointer is already installed. | |
| 6774 assert(size == 0, "Initial value"); | |
| 6775 } else { | |
| 6776 assert(p->is_parsable(), "must be parsable."); | |
| 6777 // an initialized object; ignore mark word in verification below | |
| 6778 // since we are running concurrent with mutators | |
| 6779 assert(p->is_oop(true), "should be an oop"); | |
| 6780 if (p->is_objArray()) { | |
| 6781 // objArrays are precisely marked; restrict scanning | |
| 6782 // to dirty cards only. | |
| 187 | 6783 size = CompactibleFreeListSpace::adjustObjectSize( |
| 6784 p->oop_iterate(_scanningClosure, mr)); | |
| 0 | 6785 } else { |
| 6786 // A non-array may have been imprecisely marked; we need | |
| 6787 // to scan object in its entirety. | |
| 6788 size = CompactibleFreeListSpace::adjustObjectSize( | |
| 6789 p->oop_iterate(_scanningClosure)); | |
| 6790 } | |
| 6791 #ifdef DEBUG | |
| 6792 size_t direct_size = | |
| 6793 CompactibleFreeListSpace::adjustObjectSize(p->size()); | |
| 6794 assert(size == direct_size, "Inconsistency in size"); | |
| 6795 assert(size >= 3, "Necessary for Printezis marks to work"); | |
| 6796 if (!_bitMap->isMarked(addr+1)) { | |
| 6797 _bitMap->verifyNoOneBitsInRange(addr+2, addr+size); | |
| 6798 } else { | |
| 6799 _bitMap->verifyNoOneBitsInRange(addr+2, addr+size-1); | |
| 6800 assert(_bitMap->isMarked(addr+size-1), | |
| 6801 "inconsistent Printezis mark"); | |
| 6802 } | |
| 6803 #endif // DEBUG | |
| 6804 } | |
| 6805 } else { | |
| 6806 // an unitialized object | |
| 6807 assert(_bitMap->isMarked(addr+1), "missing Printezis mark?"); | |
| 6808 HeapWord* nextOneAddr = _bitMap->getNextMarkedWordAddress(addr + 2); | |
| 6809 size = pointer_delta(nextOneAddr + 1, addr); | |
| 6810 assert(size == CompactibleFreeListSpace::adjustObjectSize(size), | |
| 6811 "alignment problem"); | |
| 6812 // Note that pre-cleaning needn't redirty the card. OopDesc::set_klass() | |
| 6813 // will dirty the card when the klass pointer is installed in the | |
| 6814 // object (signalling the completion of initialization). | |
| 6815 } | |
| 6816 } else { | |
| 6817 // Either a not yet marked object or an uninitialized object | |
| 187 | 6818 if (p->klass_or_null() == NULL || !p->is_parsable()) { |
| 0 | 6819 // An uninitialized object, skip to the next card, since |
| 6820 // we may not be able to read its P-bits yet. | |
| 6821 assert(size == 0, "Initial value"); | |
| 6822 } else { | |
| 6823 // An object not (yet) reached by marking: we merely need to | |
| 6824 // compute its size so as to go look at the next block. | |
| 6825 assert(p->is_oop(true), "should be an oop"); | |
| 6826 size = CompactibleFreeListSpace::adjustObjectSize(p->size()); | |
| 6827 } | |
| 6828 } | |
| 6829 DEBUG_ONLY(_collector->verify_work_stacks_empty();) | |
| 6830 return size; | |
| 6831 } | |
| 6832 | |
| 6833 void ScanMarkedObjectsAgainCarefullyClosure::do_yield_work() { | |
| 6834 assert(ConcurrentMarkSweepThread::cms_thread_has_cms_token(), | |
| 6835 "CMS thread should hold CMS token"); | |
| 6836 assert_lock_strong(_freelistLock); | |
| 6837 assert_lock_strong(_bitMap->lock()); | |
| 935 | 6838 DEBUG_ONLY(RememberKlassesChecker mux(false);) |
| 0 | 6839 // relinquish the free_list_lock and bitMaplock() |
| 6840 _bitMap->lock()->unlock(); | |
| 6841 _freelistLock->unlock(); | |
| 6842 ConcurrentMarkSweepThread::desynchronize(true); | |
| 6843 ConcurrentMarkSweepThread::acknowledge_yield_request(); | |
| 6844 _collector->stopTimer(); | |
| 6845 GCPauseTimer p(_collector->size_policy()->concurrent_timer_ptr()); | |
| 6846 if (PrintCMSStatistics != 0) { | |
| 6847 _collector->incrementYields(); | |
| 6848 } | |
| 6849 _collector->icms_wait(); | |
| 6850 | |
| 6851 // See the comment in coordinator_yield() | |
| 6852 for (unsigned i = 0; i < CMSYieldSleepCount && | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6853 ConcurrentMarkSweepThread::should_yield() && |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
6854 !CMSCollector::foregroundGCIsActive(); ++i) { |
| 0 | 6855 os::sleep(Thread::current(), 1, false); |
| 6856 ConcurrentMarkSweepThread::acknowledge_yield_request(); | |
| 6857 } | |
| 6858 | |
| 6859 ConcurrentMarkSweepThread::synchronize(true); | |
| 6860 _freelistLock->lock_without_safepoint_check(); | |
| 6861 _bitMap->lock()->lock_without_safepoint_check(); | |
| 6862 _collector->startTimer(); | |
| 6863 } | |
| 6864 | |
| 6865 | |
| 6866 ////////////////////////////////////////////////////////////////// | |
| 6867 // SurvivorSpacePrecleanClosure | |
| 6868 ////////////////////////////////////////////////////////////////// | |
| 6869 // This (single-threaded) closure is used to preclean the oops in | |
| 6870 // the survivor spaces. | |
| 6871 size_t SurvivorSpacePrecleanClosure::do_object_careful(oop p) { | |
| 6872 | |
| 6873 HeapWord* addr = (HeapWord*)p; | |
| 6874 DEBUG_ONLY(_collector->verify_work_stacks_empty();) | |
| 6875 assert(!_span.contains(addr), "we are scanning the survivor spaces"); | |
| 187 | 6876 assert(p->klass_or_null() != NULL, "object should be initializd"); |
| 0 | 6877 assert(p->is_parsable(), "must be parsable."); |
| 6878 // an initialized object; ignore mark word in verification below | |
| 6879 // since we are running concurrent with mutators | |
| 6880 assert(p->is_oop(true), "should be an oop"); | |
| 6881 // Note that we do not yield while we iterate over | |
| 6882 // the interior oops of p, pushing the relevant ones | |
| 6883 // on our marking stack. | |
| 6884 size_t size = p->oop_iterate(_scanning_closure); | |
| 6885 do_yield_check(); | |
| 6886 // Observe that below, we do not abandon the preclean | |
| 6887 // phase as soon as we should; rather we empty the | |
| 6888 // marking stack before returning. This is to satisfy | |
| 6889 // some existing assertions. In general, it may be a | |
| 6890 // good idea to abort immediately and complete the marking | |
| 6891 // from the grey objects at a later time. | |
| 6892 while (!_mark_stack->isEmpty()) { | |
| 6893 oop new_oop = _mark_stack->pop(); | |
| 6894 assert(new_oop != NULL && new_oop->is_oop(), "Expected an oop"); | |
| 6895 assert(new_oop->is_parsable(), "Found unparsable oop"); | |
| 6896 assert(_bit_map->isMarked((HeapWord*)new_oop), | |
| 6897 "only grey objects on this stack"); | |
| 6898 // iterate over the oops in this oop, marking and pushing | |
| 6899 // the ones in CMS heap (i.e. in _span). | |
| 6900 new_oop->oop_iterate(_scanning_closure); | |
| 6901 // check if it's time to yield | |
| 6902 do_yield_check(); | |
| 6903 } | |
| 6904 unsigned int after_count = | |
| 6905 GenCollectedHeap::heap()->total_collections(); | |
| 6906 bool abort = (_before_count != after_count) || | |
| 6907 _collector->should_abort_preclean(); | |
| 6908 return abort ? 0 : size; | |
| 6909 } | |
| 6910 | |
| 6911 void SurvivorSpacePrecleanClosure::do_yield_work() { | |
| 6912 assert(ConcurrentMarkSweepThread::cms_thread_has_cms_token(), | |
| 6913 "CMS thread should hold CMS token"); | |
| 6914 assert_lock_strong(_bit_map->lock()); | |
| 935 | 6915 DEBUG_ONLY(RememberKlassesChecker smx(false);) |
| 0 | 6916 // Relinquish the bit map lock |
| 6917 _bit_map->lock()->unlock(); | |
| 6918 ConcurrentMarkSweepThread::desynchronize(true); | |
| 6919 ConcurrentMarkSweepThread::acknowledge_yield_request(); | |
| 6920 _collector->stopTimer(); | |
| 6921 GCPauseTimer p(_collector->size_policy()->concurrent_timer_ptr()); | |
| 6922 if (PrintCMSStatistics != 0) { | |
| 6923 _collector->incrementYields(); | |
| 6924 } | |
| 6925 _collector->icms_wait(); | |
| 6926 | |
| 6927 // See the comment in coordinator_yield() | |
| 6928 for (unsigned i = 0; i < CMSYieldSleepCount && | |
| 6929 ConcurrentMarkSweepThread::should_yield() && | |
| 6930 !CMSCollector::foregroundGCIsActive(); ++i) { | |
| 6931 os::sleep(Thread::current(), 1, false); | |
| 6932 ConcurrentMarkSweepThread::acknowledge_yield_request(); | |
| 6933 } | |
| 6934 | |
| 6935 ConcurrentMarkSweepThread::synchronize(true); | |
| 6936 _bit_map->lock()->lock_without_safepoint_check(); | |
| 6937 _collector->startTimer(); | |
| 6938 } | |
| 6939 | |
| 6940 // This closure is used to rescan the marked objects on the dirty cards | |
| 6941 // in the mod union table and the card table proper. In the parallel | |
| 6942 // case, although the bitMap is shared, we do a single read so the | |
| 6943 // isMarked() query is "safe". | |
| 6944 bool ScanMarkedObjectsAgainClosure::do_object_bm(oop p, MemRegion mr) { | |
| 6945 // Ignore mark word because we are running concurrent with mutators | |
| 6946 assert(p->is_oop_or_null(true), "expected an oop or null"); | |
| 6947 HeapWord* addr = (HeapWord*)p; | |
| 6948 assert(_span.contains(addr), "we are scanning the CMS generation"); | |
| 6949 bool is_obj_array = false; | |
| 6950 #ifdef DEBUG | |
| 6951 if (!_parallel) { | |
| 6952 assert(_mark_stack->isEmpty(), "pre-condition (eager drainage)"); | |
| 6953 assert(_collector->overflow_list_is_empty(), | |
| 6954 "overflow list should be empty"); | |
| 6955 | |
| 6956 } | |
| 6957 #endif // DEBUG | |
| 6958 if (_bit_map->isMarked(addr)) { | |
| 6959 // Obj arrays are precisely marked, non-arrays are not; | |
| 6960 // so we scan objArrays precisely and non-arrays in their | |
| 6961 // entirety. | |
| 6962 if (p->is_objArray()) { | |
| 6963 is_obj_array = true; | |
| 6964 if (_parallel) { | |
| 6965 p->oop_iterate(_par_scan_closure, mr); | |
| 6966 } else { | |
| 6967 p->oop_iterate(_scan_closure, mr); | |
| 6968 } | |
| 6969 } else { | |
| 6970 if (_parallel) { | |
| 6971 p->oop_iterate(_par_scan_closure); | |
| 6972 } else { | |
| 6973 p->oop_iterate(_scan_closure); | |
| 6974 } | |
| 6975 } | |
| 6976 } | |
| 6977 #ifdef DEBUG | |
| 6978 if (!_parallel) { | |
| 6979 assert(_mark_stack->isEmpty(), "post-condition (eager drainage)"); | |
| 6980 assert(_collector->overflow_list_is_empty(), | |
| 6981 "overflow list should be empty"); | |
| 6982 | |
| 6983 } | |
| 6984 #endif // DEBUG | |
| 6985 return is_obj_array; | |
| 6986 } | |
| 6987 | |
| 6988 MarkFromRootsClosure::MarkFromRootsClosure(CMSCollector* collector, | |
| 6989 MemRegion span, | |
| 6990 CMSBitMap* bitMap, CMSMarkStack* markStack, | |
| 6991 CMSMarkStack* revisitStack, | |
| 6992 bool should_yield, bool verifying): | |
| 6993 _collector(collector), | |
| 6994 _span(span), | |
| 6995 _bitMap(bitMap), | |
| 6996 _mut(&collector->_modUnionTable), | |
| 6997 _markStack(markStack), | |
| 6998 _revisitStack(revisitStack), | |
| 6999 _yield(should_yield), | |
| 7000 _skipBits(0) | |
| 7001 { | |
| 7002 assert(_markStack->isEmpty(), "stack should be empty"); | |
| 7003 _finger = _bitMap->startWord(); | |
| 7004 _threshold = _finger; | |
| 7005 assert(_collector->_restart_addr == NULL, "Sanity check"); | |
| 7006 assert(_span.contains(_finger), "Out of bounds _finger?"); | |
| 7007 DEBUG_ONLY(_verifying = verifying;) | |
| 7008 } | |
| 7009 | |
| 7010 void MarkFromRootsClosure::reset(HeapWord* addr) { | |
| 7011 assert(_markStack->isEmpty(), "would cause duplicates on stack"); | |
| 7012 assert(_span.contains(addr), "Out of bounds _finger?"); | |
| 7013 _finger = addr; | |
| 7014 _threshold = (HeapWord*)round_to( | |
| 7015 (intptr_t)_finger, CardTableModRefBS::card_size); | |
| 7016 } | |
| 7017 | |
| 7018 // Should revisit to see if this should be restructured for | |
| 7019 // greater efficiency. | |
|
342
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
7020 bool MarkFromRootsClosure::do_bit(size_t offset) { |
| 0 | 7021 if (_skipBits > 0) { |
| 7022 _skipBits--; | |
|
342
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
7023 return true; |
| 0 | 7024 } |
| 7025 // convert offset into a HeapWord* | |
| 7026 HeapWord* addr = _bitMap->startWord() + offset; | |
| 7027 assert(_bitMap->endWord() && addr < _bitMap->endWord(), | |
| 7028 "address out of range"); | |
| 7029 assert(_bitMap->isMarked(addr), "tautology"); | |
| 7030 if (_bitMap->isMarked(addr+1)) { | |
| 7031 // this is an allocated but not yet initialized object | |
| 7032 assert(_skipBits == 0, "tautology"); | |
| 7033 _skipBits = 2; // skip next two marked bits ("Printezis-marks") | |
| 7034 oop p = oop(addr); | |
| 187 | 7035 if (p->klass_or_null() == NULL || !p->is_parsable()) { |
| 0 | 7036 DEBUG_ONLY(if (!_verifying) {) |
| 7037 // We re-dirty the cards on which this object lies and increase | |
| 7038 // the _threshold so that we'll come back to scan this object | |
| 7039 // during the preclean or remark phase. (CMSCleanOnEnter) | |
| 7040 if (CMSCleanOnEnter) { | |
| 7041 size_t sz = _collector->block_size_using_printezis_bits(addr); | |
| 7042 HeapWord* end_card_addr = (HeapWord*)round_to( | |
| 7043 (intptr_t)(addr+sz), CardTableModRefBS::card_size); | |
|
283
9199f248b0ee
6722112: CMS: Incorrect encoding of overflown object arrays during concurrent precleaning
ysr
parents:
271
diff
changeset
|
7044 MemRegion redirty_range = MemRegion(addr, end_card_addr); |
| 0 | 7045 assert(!redirty_range.is_empty(), "Arithmetical tautology"); |
| 7046 // Bump _threshold to end_card_addr; note that | |
| 7047 // _threshold cannot possibly exceed end_card_addr, anyhow. | |
| 7048 // This prevents future clearing of the card as the scan proceeds | |
| 7049 // to the right. | |
| 7050 assert(_threshold <= end_card_addr, | |
| 7051 "Because we are just scanning into this object"); | |
| 7052 if (_threshold < end_card_addr) { | |
| 7053 _threshold = end_card_addr; | |
| 7054 } | |
| 187 | 7055 if (p->klass_or_null() != NULL) { |
| 0 | 7056 // Redirty the range of cards... |
| 7057 _mut->mark_range(redirty_range); | |
| 7058 } // ...else the setting of klass will dirty the card anyway. | |
| 7059 } | |
| 7060 DEBUG_ONLY(}) | |
|
342
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
7061 return true; |
| 0 | 7062 } |
| 7063 } | |
| 7064 scanOopsInOop(addr); | |
|
342
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
7065 return true; |
| 0 | 7066 } |
| 7067 | |
| 7068 // We take a break if we've been at this for a while, | |
| 7069 // so as to avoid monopolizing the locks involved. | |
| 7070 void MarkFromRootsClosure::do_yield_work() { | |
| 7071 // First give up the locks, then yield, then re-lock | |
| 7072 // We should probably use a constructor/destructor idiom to | |
| 7073 // do this unlock/lock or modify the MutexUnlocker class to | |
| 7074 // serve our purpose. XXX | |
| 7075 assert(ConcurrentMarkSweepThread::cms_thread_has_cms_token(), | |
| 7076 "CMS thread should hold CMS token"); | |
| 7077 assert_lock_strong(_bitMap->lock()); | |
| 935 | 7078 DEBUG_ONLY(RememberKlassesChecker mux(false);) |
| 0 | 7079 _bitMap->lock()->unlock(); |
| 7080 ConcurrentMarkSweepThread::desynchronize(true); | |
| 7081 ConcurrentMarkSweepThread::acknowledge_yield_request(); | |
| 7082 _collector->stopTimer(); | |
| 7083 GCPauseTimer p(_collector->size_policy()->concurrent_timer_ptr()); | |
| 7084 if (PrintCMSStatistics != 0) { | |
| 7085 _collector->incrementYields(); | |
| 7086 } | |
| 7087 _collector->icms_wait(); | |
| 7088 | |
| 7089 // See the comment in coordinator_yield() | |
| 7090 for (unsigned i = 0; i < CMSYieldSleepCount && | |
| 7091 ConcurrentMarkSweepThread::should_yield() && | |
| 7092 !CMSCollector::foregroundGCIsActive(); ++i) { | |
| 7093 os::sleep(Thread::current(), 1, false); | |
| 7094 ConcurrentMarkSweepThread::acknowledge_yield_request(); | |
| 7095 } | |
| 7096 | |
| 7097 ConcurrentMarkSweepThread::synchronize(true); | |
| 7098 _bitMap->lock()->lock_without_safepoint_check(); | |
| 7099 _collector->startTimer(); | |
| 7100 } | |
| 7101 | |
| 7102 void MarkFromRootsClosure::scanOopsInOop(HeapWord* ptr) { | |
| 7103 assert(_bitMap->isMarked(ptr), "expected bit to be set"); | |
| 7104 assert(_markStack->isEmpty(), | |
| 7105 "should drain stack to limit stack usage"); | |
| 7106 // convert ptr to an oop preparatory to scanning | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7107 oop obj = oop(ptr); |
| 0 | 7108 // Ignore mark word in verification below, since we |
| 7109 // may be running concurrent with mutators. | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7110 assert(obj->is_oop(true), "should be an oop"); |
| 0 | 7111 assert(_finger <= ptr, "_finger runneth ahead"); |
| 7112 // advance the finger to right end of this object | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7113 _finger = ptr + obj->size(); |
| 0 | 7114 assert(_finger > ptr, "we just incremented it above"); |
| 7115 // On large heaps, it may take us some time to get through | |
| 7116 // the marking phase (especially if running iCMS). During | |
| 7117 // this time it's possible that a lot of mutations have | |
| 7118 // accumulated in the card table and the mod union table -- | |
| 7119 // these mutation records are redundant until we have | |
| 7120 // actually traced into the corresponding card. | |
| 7121 // Here, we check whether advancing the finger would make | |
| 7122 // us cross into a new card, and if so clear corresponding | |
| 7123 // cards in the MUT (preclean them in the card-table in the | |
| 7124 // future). | |
| 7125 | |
| 7126 DEBUG_ONLY(if (!_verifying) {) | |
| 7127 // The clean-on-enter optimization is disabled by default, | |
| 7128 // until we fix 6178663. | |
| 7129 if (CMSCleanOnEnter && (_finger > _threshold)) { | |
| 7130 // [_threshold, _finger) represents the interval | |
| 7131 // of cards to be cleared in MUT (or precleaned in card table). | |
| 7132 // The set of cards to be cleared is all those that overlap | |
| 7133 // with the interval [_threshold, _finger); note that | |
| 7134 // _threshold is always kept card-aligned but _finger isn't | |
| 7135 // always card-aligned. | |
| 7136 HeapWord* old_threshold = _threshold; | |
| 7137 assert(old_threshold == (HeapWord*)round_to( | |
| 7138 (intptr_t)old_threshold, CardTableModRefBS::card_size), | |
| 7139 "_threshold should always be card-aligned"); | |
| 7140 _threshold = (HeapWord*)round_to( | |
| 7141 (intptr_t)_finger, CardTableModRefBS::card_size); | |
| 7142 MemRegion mr(old_threshold, _threshold); | |
| 7143 assert(!mr.is_empty(), "Control point invariant"); | |
| 7144 assert(_span.contains(mr), "Should clear within span"); | |
| 7145 // XXX When _finger crosses from old gen into perm gen | |
| 7146 // we may be doing unnecessary cleaning; do better in the | |
| 7147 // future by detecting that condition and clearing fewer | |
| 7148 // MUT/CT entries. | |
| 7149 _mut->clear_range(mr); | |
| 7150 } | |
| 7151 DEBUG_ONLY(}) | |
| 7152 // Note: the finger doesn't advance while we drain | |
| 7153 // the stack below. | |
| 7154 PushOrMarkClosure pushOrMarkClosure(_collector, | |
| 7155 _span, _bitMap, _markStack, | |
| 7156 _revisitStack, | |
| 7157 _finger, this); | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7158 bool res = _markStack->push(obj); |
| 0 | 7159 assert(res, "Empty non-zero size stack should have space for single push"); |
| 7160 while (!_markStack->isEmpty()) { | |
| 7161 oop new_oop = _markStack->pop(); | |
| 7162 // Skip verifying header mark word below because we are | |
| 7163 // running concurrent with mutators. | |
| 7164 assert(new_oop->is_oop(true), "Oops! expected to pop an oop"); | |
| 7165 // now scan this oop's oops | |
| 7166 new_oop->oop_iterate(&pushOrMarkClosure); | |
| 7167 do_yield_check(); | |
| 7168 } | |
| 7169 assert(_markStack->isEmpty(), "tautology, emphasizing post-condition"); | |
| 7170 } | |
| 7171 | |
| 7172 Par_MarkFromRootsClosure::Par_MarkFromRootsClosure(CMSConcMarkingTask* task, | |
| 7173 CMSCollector* collector, MemRegion span, | |
| 7174 CMSBitMap* bit_map, | |
| 7175 OopTaskQueue* work_queue, | |
| 7176 CMSMarkStack* overflow_stack, | |
| 7177 CMSMarkStack* revisit_stack, | |
| 7178 bool should_yield): | |
| 7179 _collector(collector), | |
| 7180 _whole_span(collector->_span), | |
| 7181 _span(span), | |
| 7182 _bit_map(bit_map), | |
| 7183 _mut(&collector->_modUnionTable), | |
| 7184 _work_queue(work_queue), | |
| 7185 _overflow_stack(overflow_stack), | |
| 7186 _revisit_stack(revisit_stack), | |
| 7187 _yield(should_yield), | |
| 7188 _skip_bits(0), | |
| 7189 _task(task) | |
| 7190 { | |
| 7191 assert(_work_queue->size() == 0, "work_queue should be empty"); | |
| 7192 _finger = span.start(); | |
| 7193 _threshold = _finger; // XXX Defer clear-on-enter optimization for now | |
| 7194 assert(_span.contains(_finger), "Out of bounds _finger?"); | |
| 7195 } | |
| 7196 | |
| 7197 // Should revisit to see if this should be restructured for | |
| 7198 // greater efficiency. | |
|
342
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
7199 bool Par_MarkFromRootsClosure::do_bit(size_t offset) { |
| 0 | 7200 if (_skip_bits > 0) { |
| 7201 _skip_bits--; | |
|
342
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
7202 return true; |
| 0 | 7203 } |
| 7204 // convert offset into a HeapWord* | |
| 7205 HeapWord* addr = _bit_map->startWord() + offset; | |
| 7206 assert(_bit_map->endWord() && addr < _bit_map->endWord(), | |
| 7207 "address out of range"); | |
| 7208 assert(_bit_map->isMarked(addr), "tautology"); | |
| 7209 if (_bit_map->isMarked(addr+1)) { | |
| 7210 // this is an allocated object that might not yet be initialized | |
| 7211 assert(_skip_bits == 0, "tautology"); | |
| 7212 _skip_bits = 2; // skip next two marked bits ("Printezis-marks") | |
| 7213 oop p = oop(addr); | |
| 187 | 7214 if (p->klass_or_null() == NULL || !p->is_parsable()) { |
| 0 | 7215 // in the case of Clean-on-Enter optimization, redirty card |
| 7216 // and avoid clearing card by increasing the threshold. | |
|
342
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
7217 return true; |
| 0 | 7218 } |
| 7219 } | |
| 7220 scan_oops_in_oop(addr); | |
|
342
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
7221 return true; |
| 0 | 7222 } |
| 7223 | |
| 7224 void Par_MarkFromRootsClosure::scan_oops_in_oop(HeapWord* ptr) { | |
| 7225 assert(_bit_map->isMarked(ptr), "expected bit to be set"); | |
| 7226 // Should we assert that our work queue is empty or | |
| 7227 // below some drain limit? | |
| 7228 assert(_work_queue->size() == 0, | |
| 7229 "should drain stack to limit stack usage"); | |
| 7230 // convert ptr to an oop preparatory to scanning | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7231 oop obj = oop(ptr); |
| 0 | 7232 // Ignore mark word in verification below, since we |
| 7233 // may be running concurrent with mutators. | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7234 assert(obj->is_oop(true), "should be an oop"); |
| 0 | 7235 assert(_finger <= ptr, "_finger runneth ahead"); |
| 7236 // advance the finger to right end of this object | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7237 _finger = ptr + obj->size(); |
| 0 | 7238 assert(_finger > ptr, "we just incremented it above"); |
| 7239 // On large heaps, it may take us some time to get through | |
| 7240 // the marking phase (especially if running iCMS). During | |
| 7241 // this time it's possible that a lot of mutations have | |
| 7242 // accumulated in the card table and the mod union table -- | |
| 7243 // these mutation records are redundant until we have | |
| 7244 // actually traced into the corresponding card. | |
| 7245 // Here, we check whether advancing the finger would make | |
| 7246 // us cross into a new card, and if so clear corresponding | |
| 7247 // cards in the MUT (preclean them in the card-table in the | |
| 7248 // future). | |
| 7249 | |
| 7250 // The clean-on-enter optimization is disabled by default, | |
| 7251 // until we fix 6178663. | |
| 7252 if (CMSCleanOnEnter && (_finger > _threshold)) { | |
| 7253 // [_threshold, _finger) represents the interval | |
| 7254 // of cards to be cleared in MUT (or precleaned in card table). | |
| 7255 // The set of cards to be cleared is all those that overlap | |
| 7256 // with the interval [_threshold, _finger); note that | |
| 7257 // _threshold is always kept card-aligned but _finger isn't | |
| 7258 // always card-aligned. | |
| 7259 HeapWord* old_threshold = _threshold; | |
| 7260 assert(old_threshold == (HeapWord*)round_to( | |
| 7261 (intptr_t)old_threshold, CardTableModRefBS::card_size), | |
| 7262 "_threshold should always be card-aligned"); | |
| 7263 _threshold = (HeapWord*)round_to( | |
| 7264 (intptr_t)_finger, CardTableModRefBS::card_size); | |
| 7265 MemRegion mr(old_threshold, _threshold); | |
| 7266 assert(!mr.is_empty(), "Control point invariant"); | |
| 7267 assert(_span.contains(mr), "Should clear within span"); // _whole_span ?? | |
| 7268 // XXX When _finger crosses from old gen into perm gen | |
| 7269 // we may be doing unnecessary cleaning; do better in the | |
| 7270 // future by detecting that condition and clearing fewer | |
| 7271 // MUT/CT entries. | |
| 7272 _mut->clear_range(mr); | |
| 7273 } | |
| 7274 | |
| 7275 // Note: the local finger doesn't advance while we drain | |
| 7276 // the stack below, but the global finger sure can and will. | |
| 7277 HeapWord** gfa = _task->global_finger_addr(); | |
| 7278 Par_PushOrMarkClosure pushOrMarkClosure(_collector, | |
| 7279 _span, _bit_map, | |
| 7280 _work_queue, | |
| 7281 _overflow_stack, | |
| 7282 _revisit_stack, | |
| 7283 _finger, | |
| 7284 gfa, this); | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7285 bool res = _work_queue->push(obj); // overflow could occur here |
| 0 | 7286 assert(res, "Will hold once we use workqueues"); |
| 7287 while (true) { | |
| 7288 oop new_oop; | |
| 7289 if (!_work_queue->pop_local(new_oop)) { | |
| 7290 // We emptied our work_queue; check if there's stuff that can | |
| 7291 // be gotten from the overflow stack. | |
| 7292 if (CMSConcMarkingTask::get_work_from_overflow_stack( | |
| 7293 _overflow_stack, _work_queue)) { | |
| 7294 do_yield_check(); | |
| 7295 continue; | |
| 7296 } else { // done | |
| 7297 break; | |
| 7298 } | |
| 7299 } | |
| 7300 // Skip verifying header mark word below because we are | |
| 7301 // running concurrent with mutators. | |
| 7302 assert(new_oop->is_oop(true), "Oops! expected to pop an oop"); | |
| 7303 // now scan this oop's oops | |
| 7304 new_oop->oop_iterate(&pushOrMarkClosure); | |
| 7305 do_yield_check(); | |
| 7306 } | |
| 7307 assert(_work_queue->size() == 0, "tautology, emphasizing post-condition"); | |
| 7308 } | |
| 7309 | |
| 7310 // Yield in response to a request from VM Thread or | |
| 7311 // from mutators. | |
| 7312 void Par_MarkFromRootsClosure::do_yield_work() { | |
| 7313 assert(_task != NULL, "sanity"); | |
| 7314 _task->yield(); | |
| 7315 } | |
| 7316 | |
| 7317 // A variant of the above used for verifying CMS marking work. | |
| 7318 MarkFromRootsVerifyClosure::MarkFromRootsVerifyClosure(CMSCollector* collector, | |
| 7319 MemRegion span, | |
| 7320 CMSBitMap* verification_bm, CMSBitMap* cms_bm, | |
| 7321 CMSMarkStack* mark_stack): | |
| 7322 _collector(collector), | |
| 7323 _span(span), | |
| 7324 _verification_bm(verification_bm), | |
| 7325 _cms_bm(cms_bm), | |
| 7326 _mark_stack(mark_stack), | |
| 7327 _pam_verify_closure(collector, span, verification_bm, cms_bm, | |
| 7328 mark_stack) | |
| 7329 { | |
| 7330 assert(_mark_stack->isEmpty(), "stack should be empty"); | |
| 7331 _finger = _verification_bm->startWord(); | |
| 7332 assert(_collector->_restart_addr == NULL, "Sanity check"); | |
| 7333 assert(_span.contains(_finger), "Out of bounds _finger?"); | |
| 7334 } | |
| 7335 | |
| 7336 void MarkFromRootsVerifyClosure::reset(HeapWord* addr) { | |
| 7337 assert(_mark_stack->isEmpty(), "would cause duplicates on stack"); | |
| 7338 assert(_span.contains(addr), "Out of bounds _finger?"); | |
| 7339 _finger = addr; | |
| 7340 } | |
| 7341 | |
| 7342 // Should revisit to see if this should be restructured for | |
| 7343 // greater efficiency. | |
|
342
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
7344 bool MarkFromRootsVerifyClosure::do_bit(size_t offset) { |
| 0 | 7345 // convert offset into a HeapWord* |
| 7346 HeapWord* addr = _verification_bm->startWord() + offset; | |
| 7347 assert(_verification_bm->endWord() && addr < _verification_bm->endWord(), | |
| 7348 "address out of range"); | |
| 7349 assert(_verification_bm->isMarked(addr), "tautology"); | |
| 7350 assert(_cms_bm->isMarked(addr), "tautology"); | |
| 7351 | |
| 7352 assert(_mark_stack->isEmpty(), | |
| 7353 "should drain stack to limit stack usage"); | |
| 7354 // convert addr to an oop preparatory to scanning | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7355 oop obj = oop(addr); |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7356 assert(obj->is_oop(), "should be an oop"); |
| 0 | 7357 assert(_finger <= addr, "_finger runneth ahead"); |
| 7358 // advance the finger to right end of this object | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7359 _finger = addr + obj->size(); |
| 0 | 7360 assert(_finger > addr, "we just incremented it above"); |
| 7361 // Note: the finger doesn't advance while we drain | |
| 7362 // the stack below. | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7363 bool res = _mark_stack->push(obj); |
| 0 | 7364 assert(res, "Empty non-zero size stack should have space for single push"); |
| 7365 while (!_mark_stack->isEmpty()) { | |
| 7366 oop new_oop = _mark_stack->pop(); | |
| 7367 assert(new_oop->is_oop(), "Oops! expected to pop an oop"); | |
| 7368 // now scan this oop's oops | |
| 7369 new_oop->oop_iterate(&_pam_verify_closure); | |
| 7370 } | |
| 7371 assert(_mark_stack->isEmpty(), "tautology, emphasizing post-condition"); | |
|
342
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
7372 return true; |
| 0 | 7373 } |
| 7374 | |
| 7375 PushAndMarkVerifyClosure::PushAndMarkVerifyClosure( | |
| 7376 CMSCollector* collector, MemRegion span, | |
| 7377 CMSBitMap* verification_bm, CMSBitMap* cms_bm, | |
| 7378 CMSMarkStack* mark_stack): | |
| 7379 OopClosure(collector->ref_processor()), | |
| 7380 _collector(collector), | |
| 7381 _span(span), | |
| 7382 _verification_bm(verification_bm), | |
| 7383 _cms_bm(cms_bm), | |
| 7384 _mark_stack(mark_stack) | |
| 7385 { } | |
| 7386 | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7387 void PushAndMarkVerifyClosure::do_oop(oop* p) { PushAndMarkVerifyClosure::do_oop_work(p); } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7388 void PushAndMarkVerifyClosure::do_oop(narrowOop* p) { PushAndMarkVerifyClosure::do_oop_work(p); } |
| 0 | 7389 |
| 7390 // Upon stack overflow, we discard (part of) the stack, | |
| 7391 // remembering the least address amongst those discarded | |
| 7392 // in CMSCollector's _restart_address. | |
| 7393 void PushAndMarkVerifyClosure::handle_stack_overflow(HeapWord* lost) { | |
| 7394 // Remember the least grey address discarded | |
| 7395 HeapWord* ra = (HeapWord*)_mark_stack->least_value(lost); | |
| 7396 _collector->lower_restart_addr(ra); | |
| 7397 _mark_stack->reset(); // discard stack contents | |
| 7398 _mark_stack->expand(); // expand the stack if possible | |
| 7399 } | |
| 7400 | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7401 void PushAndMarkVerifyClosure::do_oop(oop obj) { |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7402 assert(obj->is_oop_or_null(), "expected an oop or NULL"); |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7403 HeapWord* addr = (HeapWord*)obj; |
| 0 | 7404 if (_span.contains(addr) && !_verification_bm->isMarked(addr)) { |
| 7405 // Oop lies in _span and isn't yet grey or black | |
| 7406 _verification_bm->mark(addr); // now grey | |
| 7407 if (!_cms_bm->isMarked(addr)) { | |
| 7408 oop(addr)->print(); | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7409 gclog_or_tty->print_cr(" (" INTPTR_FORMAT " should have been marked)", |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7410 addr); |
| 0 | 7411 fatal("... aborting"); |
| 7412 } | |
| 7413 | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7414 if (!_mark_stack->push(obj)) { // stack overflow |
| 0 | 7415 if (PrintCMSStatistics != 0) { |
| 7416 gclog_or_tty->print_cr("CMS marking stack overflow (benign) at " | |
| 7417 SIZE_FORMAT, _mark_stack->capacity()); | |
| 7418 } | |
| 7419 assert(_mark_stack->isFull(), "Else push should have succeeded"); | |
| 7420 handle_stack_overflow(addr); | |
| 7421 } | |
| 7422 // anything including and to the right of _finger | |
| 7423 // will be scanned as we iterate over the remainder of the | |
| 7424 // bit map | |
| 7425 } | |
| 7426 } | |
| 7427 | |
| 7428 PushOrMarkClosure::PushOrMarkClosure(CMSCollector* collector, | |
| 7429 MemRegion span, | |
| 7430 CMSBitMap* bitMap, CMSMarkStack* markStack, | |
| 7431 CMSMarkStack* revisitStack, | |
| 7432 HeapWord* finger, MarkFromRootsClosure* parent) : | |
| 935 | 7433 KlassRememberingOopClosure(collector, collector->ref_processor(), revisitStack), |
| 0 | 7434 _span(span), |
| 7435 _bitMap(bitMap), | |
| 7436 _markStack(markStack), | |
| 7437 _finger(finger), | |
| 935 | 7438 _parent(parent) |
| 0 | 7439 { } |
| 7440 | |
| 7441 Par_PushOrMarkClosure::Par_PushOrMarkClosure(CMSCollector* collector, | |
| 7442 MemRegion span, | |
| 7443 CMSBitMap* bit_map, | |
| 7444 OopTaskQueue* work_queue, | |
| 7445 CMSMarkStack* overflow_stack, | |
| 7446 CMSMarkStack* revisit_stack, | |
| 7447 HeapWord* finger, | |
| 7448 HeapWord** global_finger_addr, | |
| 7449 Par_MarkFromRootsClosure* parent) : | |
| 935 | 7450 Par_KlassRememberingOopClosure(collector, |
| 7451 collector->ref_processor(), | |
| 7452 revisit_stack), | |
| 0 | 7453 _whole_span(collector->_span), |
| 7454 _span(span), | |
| 7455 _bit_map(bit_map), | |
| 7456 _work_queue(work_queue), | |
| 7457 _overflow_stack(overflow_stack), | |
| 7458 _finger(finger), | |
| 7459 _global_finger_addr(global_finger_addr), | |
| 935 | 7460 _parent(parent) |
| 0 | 7461 { } |
| 7462 | |
|
340
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
7463 // Assumes thread-safe access by callers, who are |
|
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
7464 // responsible for mutual exclusion. |
| 0 | 7465 void CMSCollector::lower_restart_addr(HeapWord* low) { |
| 7466 assert(_span.contains(low), "Out of bounds addr"); | |
| 7467 if (_restart_addr == NULL) { | |
| 7468 _restart_addr = low; | |
| 7469 } else { | |
| 7470 _restart_addr = MIN2(_restart_addr, low); | |
| 7471 } | |
| 7472 } | |
| 7473 | |
| 7474 // Upon stack overflow, we discard (part of) the stack, | |
| 7475 // remembering the least address amongst those discarded | |
| 7476 // in CMSCollector's _restart_address. | |
| 7477 void PushOrMarkClosure::handle_stack_overflow(HeapWord* lost) { | |
| 7478 // Remember the least grey address discarded | |
| 7479 HeapWord* ra = (HeapWord*)_markStack->least_value(lost); | |
| 7480 _collector->lower_restart_addr(ra); | |
| 7481 _markStack->reset(); // discard stack contents | |
| 7482 _markStack->expand(); // expand the stack if possible | |
| 7483 } | |
| 7484 | |
| 7485 // Upon stack overflow, we discard (part of) the stack, | |
| 7486 // remembering the least address amongst those discarded | |
| 7487 // in CMSCollector's _restart_address. | |
| 7488 void Par_PushOrMarkClosure::handle_stack_overflow(HeapWord* lost) { | |
| 7489 // We need to do this under a mutex to prevent other | |
|
340
ebeb6490b814
6722116: CMS: Incorrect overflow handling when using parallel concurrent marking
ysr
parents:
283
diff
changeset
|
7490 // workers from interfering with the work done below. |
| 0 | 7491 MutexLockerEx ml(_overflow_stack->par_lock(), |
| 7492 Mutex::_no_safepoint_check_flag); | |
| 7493 // Remember the least grey address discarded | |
| 7494 HeapWord* ra = (HeapWord*)_overflow_stack->least_value(lost); | |
| 7495 _collector->lower_restart_addr(ra); | |
| 7496 _overflow_stack->reset(); // discard stack contents | |
| 7497 _overflow_stack->expand(); // expand the stack if possible | |
| 7498 } | |
| 7499 | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7500 void PushOrMarkClosure::do_oop(oop obj) { |
| 0 | 7501 // Ignore mark word because we are running concurrent with mutators. |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7502 assert(obj->is_oop_or_null(true), "expected an oop or NULL"); |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7503 HeapWord* addr = (HeapWord*)obj; |
| 0 | 7504 if (_span.contains(addr) && !_bitMap->isMarked(addr)) { |
| 7505 // Oop lies in _span and isn't yet grey or black | |
| 7506 _bitMap->mark(addr); // now grey | |
| 7507 if (addr < _finger) { | |
| 7508 // the bit map iteration has already either passed, or | |
| 7509 // sampled, this bit in the bit map; we'll need to | |
| 7510 // use the marking stack to scan this oop's oops. | |
| 7511 bool simulate_overflow = false; | |
| 7512 NOT_PRODUCT( | |
| 7513 if (CMSMarkStackOverflowALot && | |
| 7514 _collector->simulate_overflow()) { | |
| 7515 // simulate a stack overflow | |
| 7516 simulate_overflow = true; | |
| 7517 } | |
| 7518 ) | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7519 if (simulate_overflow || !_markStack->push(obj)) { // stack overflow |
| 0 | 7520 if (PrintCMSStatistics != 0) { |
| 7521 gclog_or_tty->print_cr("CMS marking stack overflow (benign) at " | |
| 7522 SIZE_FORMAT, _markStack->capacity()); | |
| 7523 } | |
| 7524 assert(simulate_overflow || _markStack->isFull(), "Else push should have succeeded"); | |
| 7525 handle_stack_overflow(addr); | |
| 7526 } | |
| 7527 } | |
| 7528 // anything including and to the right of _finger | |
| 7529 // will be scanned as we iterate over the remainder of the | |
| 7530 // bit map | |
| 7531 do_yield_check(); | |
| 7532 } | |
| 7533 } | |
| 7534 | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7535 void PushOrMarkClosure::do_oop(oop* p) { PushOrMarkClosure::do_oop_work(p); } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7536 void PushOrMarkClosure::do_oop(narrowOop* p) { PushOrMarkClosure::do_oop_work(p); } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7537 |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7538 void Par_PushOrMarkClosure::do_oop(oop obj) { |
| 0 | 7539 // Ignore mark word because we are running concurrent with mutators. |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7540 assert(obj->is_oop_or_null(true), "expected an oop or NULL"); |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7541 HeapWord* addr = (HeapWord*)obj; |
| 0 | 7542 if (_whole_span.contains(addr) && !_bit_map->isMarked(addr)) { |
| 7543 // Oop lies in _span and isn't yet grey or black | |
| 7544 // We read the global_finger (volatile read) strictly after marking oop | |
| 7545 bool res = _bit_map->par_mark(addr); // now grey | |
| 7546 volatile HeapWord** gfa = (volatile HeapWord**)_global_finger_addr; | |
| 7547 // Should we push this marked oop on our stack? | |
| 7548 // -- if someone else marked it, nothing to do | |
| 7549 // -- if target oop is above global finger nothing to do | |
| 7550 // -- if target oop is in chunk and above local finger | |
| 7551 // then nothing to do | |
| 7552 // -- else push on work queue | |
| 7553 if ( !res // someone else marked it, they will deal with it | |
| 7554 || (addr >= *gfa) // will be scanned in a later task | |
| 7555 || (_span.contains(addr) && addr >= _finger)) { // later in this chunk | |
| 7556 return; | |
| 7557 } | |
| 7558 // the bit map iteration has already either passed, or | |
| 7559 // sampled, this bit in the bit map; we'll need to | |
| 7560 // use the marking stack to scan this oop's oops. | |
| 7561 bool simulate_overflow = false; | |
| 7562 NOT_PRODUCT( | |
| 7563 if (CMSMarkStackOverflowALot && | |
| 7564 _collector->simulate_overflow()) { | |
| 7565 // simulate a stack overflow | |
| 7566 simulate_overflow = true; | |
| 7567 } | |
| 7568 ) | |
| 7569 if (simulate_overflow || | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7570 !(_work_queue->push(obj) || _overflow_stack->par_push(obj))) { |
| 0 | 7571 // stack overflow |
| 7572 if (PrintCMSStatistics != 0) { | |
| 7573 gclog_or_tty->print_cr("CMS marking stack overflow (benign) at " | |
| 7574 SIZE_FORMAT, _overflow_stack->capacity()); | |
| 7575 } | |
| 7576 // We cannot assert that the overflow stack is full because | |
| 7577 // it may have been emptied since. | |
| 7578 assert(simulate_overflow || | |
| 7579 _work_queue->size() == _work_queue->max_elems(), | |
| 7580 "Else push should have succeeded"); | |
| 7581 handle_stack_overflow(addr); | |
| 7582 } | |
| 7583 do_yield_check(); | |
| 7584 } | |
| 7585 } | |
| 7586 | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7587 void Par_PushOrMarkClosure::do_oop(oop* p) { Par_PushOrMarkClosure::do_oop_work(p); } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7588 void Par_PushOrMarkClosure::do_oop(narrowOop* p) { Par_PushOrMarkClosure::do_oop_work(p); } |
| 0 | 7589 |
| 935 | 7590 KlassRememberingOopClosure::KlassRememberingOopClosure(CMSCollector* collector, |
| 7591 ReferenceProcessor* rp, | |
| 7592 CMSMarkStack* revisit_stack) : | |
| 7593 OopClosure(rp), | |
| 7594 _collector(collector), | |
| 7595 _revisit_stack(revisit_stack), | |
| 7596 _should_remember_klasses(collector->should_unload_classes()) {} | |
| 7597 | |
| 0 | 7598 PushAndMarkClosure::PushAndMarkClosure(CMSCollector* collector, |
| 7599 MemRegion span, | |
| 7600 ReferenceProcessor* rp, | |
| 7601 CMSBitMap* bit_map, | |
| 7602 CMSBitMap* mod_union_table, | |
| 7603 CMSMarkStack* mark_stack, | |
| 7604 CMSMarkStack* revisit_stack, | |
| 7605 bool concurrent_precleaning): | |
| 935 | 7606 KlassRememberingOopClosure(collector, rp, revisit_stack), |
| 0 | 7607 _span(span), |
| 7608 _bit_map(bit_map), | |
| 7609 _mod_union_table(mod_union_table), | |
| 7610 _mark_stack(mark_stack), | |
| 935 | 7611 _concurrent_precleaning(concurrent_precleaning) |
| 0 | 7612 { |
| 7613 assert(_ref_processor != NULL, "_ref_processor shouldn't be NULL"); | |
| 7614 } | |
| 7615 | |
| 7616 // Grey object rescan during pre-cleaning and second checkpoint phases -- | |
| 7617 // the non-parallel version (the parallel version appears further below.) | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7618 void PushAndMarkClosure::do_oop(oop obj) { |
|
342
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
7619 // Ignore mark word verification. If during concurrent precleaning, |
|
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
7620 // the object monitor may be locked. If during the checkpoint |
|
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
7621 // phases, the object may already have been reached by a different |
|
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
7622 // path and may be at the end of the global overflow list (so |
|
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
7623 // the mark word may be NULL). |
|
37f87013dfd8
6711316: Open source the Garbage-First garbage collector
ysr
parents:
143
diff
changeset
|
7624 assert(obj->is_oop_or_null(true /* ignore mark word */), |
| 0 | 7625 "expected an oop or NULL"); |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7626 HeapWord* addr = (HeapWord*)obj; |
| 0 | 7627 // Check if oop points into the CMS generation |
| 7628 // and is not marked | |
| 7629 if (_span.contains(addr) && !_bit_map->isMarked(addr)) { | |
| 7630 // a white object ... | |
| 7631 _bit_map->mark(addr); // ... now grey | |
| 7632 // push on the marking stack (grey set) | |
| 7633 bool simulate_overflow = false; | |
| 7634 NOT_PRODUCT( | |
| 7635 if (CMSMarkStackOverflowALot && | |
| 7636 _collector->simulate_overflow()) { | |
| 7637 // simulate a stack overflow | |
| 7638 simulate_overflow = true; | |
| 7639 } | |
| 7640 ) | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7641 if (simulate_overflow || !_mark_stack->push(obj)) { |
| 0 | 7642 if (_concurrent_precleaning) { |
|
283
9199f248b0ee
6722112: CMS: Incorrect encoding of overflown object arrays during concurrent precleaning
ysr
parents:
271
diff
changeset
|
7643 // During precleaning we can just dirty the appropriate card(s) |
| 0 | 7644 // in the mod union table, thus ensuring that the object remains |
|
283
9199f248b0ee
6722112: CMS: Incorrect encoding of overflown object arrays during concurrent precleaning
ysr
parents:
271
diff
changeset
|
7645 // in the grey set and continue. In the case of object arrays |
|
9199f248b0ee
6722112: CMS: Incorrect encoding of overflown object arrays during concurrent precleaning
ysr
parents:
271
diff
changeset
|
7646 // we need to dirty all of the cards that the object spans, |
|
9199f248b0ee
6722112: CMS: Incorrect encoding of overflown object arrays during concurrent precleaning
ysr
parents:
271
diff
changeset
|
7647 // since the rescan of object arrays will be limited to the |
|
9199f248b0ee
6722112: CMS: Incorrect encoding of overflown object arrays during concurrent precleaning
ysr
parents:
271
diff
changeset
|
7648 // dirty cards. |
|
9199f248b0ee
6722112: CMS: Incorrect encoding of overflown object arrays during concurrent precleaning
ysr
parents:
271
diff
changeset
|
7649 // Note that no one can be intefering with us in this action |
|
9199f248b0ee
6722112: CMS: Incorrect encoding of overflown object arrays during concurrent precleaning
ysr
parents:
271
diff
changeset
|
7650 // of dirtying the mod union table, so no locking or atomics |
|
9199f248b0ee
6722112: CMS: Incorrect encoding of overflown object arrays during concurrent precleaning
ysr
parents:
271
diff
changeset
|
7651 // are required. |
|
9199f248b0ee
6722112: CMS: Incorrect encoding of overflown object arrays during concurrent precleaning
ysr
parents:
271
diff
changeset
|
7652 if (obj->is_objArray()) { |
|
9199f248b0ee
6722112: CMS: Incorrect encoding of overflown object arrays during concurrent precleaning
ysr
parents:
271
diff
changeset
|
7653 size_t sz = obj->size(); |
|
9199f248b0ee
6722112: CMS: Incorrect encoding of overflown object arrays during concurrent precleaning
ysr
parents:
271
diff
changeset
|
7654 HeapWord* end_card_addr = (HeapWord*)round_to( |
|
9199f248b0ee
6722112: CMS: Incorrect encoding of overflown object arrays during concurrent precleaning
ysr
parents:
271
diff
changeset
|
7655 (intptr_t)(addr+sz), CardTableModRefBS::card_size); |
|
9199f248b0ee
6722112: CMS: Incorrect encoding of overflown object arrays during concurrent precleaning
ysr
parents:
271
diff
changeset
|
7656 MemRegion redirty_range = MemRegion(addr, end_card_addr); |
|
9199f248b0ee
6722112: CMS: Incorrect encoding of overflown object arrays during concurrent precleaning
ysr
parents:
271
diff
changeset
|
7657 assert(!redirty_range.is_empty(), "Arithmetical tautology"); |
|
9199f248b0ee
6722112: CMS: Incorrect encoding of overflown object arrays during concurrent precleaning
ysr
parents:
271
diff
changeset
|
7658 _mod_union_table->mark_range(redirty_range); |
|
9199f248b0ee
6722112: CMS: Incorrect encoding of overflown object arrays during concurrent precleaning
ysr
parents:
271
diff
changeset
|
7659 } else { |
|
9199f248b0ee
6722112: CMS: Incorrect encoding of overflown object arrays during concurrent precleaning
ysr
parents:
271
diff
changeset
|
7660 _mod_union_table->mark(addr); |
|
9199f248b0ee
6722112: CMS: Incorrect encoding of overflown object arrays during concurrent precleaning
ysr
parents:
271
diff
changeset
|
7661 } |
| 0 | 7662 _collector->_ser_pmc_preclean_ovflw++; |
| 7663 } else { | |
| 7664 // During the remark phase, we need to remember this oop | |
| 7665 // in the overflow list. | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7666 _collector->push_on_overflow_list(obj); |
| 0 | 7667 _collector->_ser_pmc_remark_ovflw++; |
| 7668 } | |
| 7669 } | |
| 7670 } | |
| 7671 } | |
| 7672 | |
| 7673 Par_PushAndMarkClosure::Par_PushAndMarkClosure(CMSCollector* collector, | |
| 7674 MemRegion span, | |
| 7675 ReferenceProcessor* rp, | |
| 7676 CMSBitMap* bit_map, | |
| 7677 OopTaskQueue* work_queue, | |
| 7678 CMSMarkStack* revisit_stack): | |
| 935 | 7679 Par_KlassRememberingOopClosure(collector, rp, revisit_stack), |
| 0 | 7680 _span(span), |
| 7681 _bit_map(bit_map), | |
| 935 | 7682 _work_queue(work_queue) |
| 0 | 7683 { |
| 7684 assert(_ref_processor != NULL, "_ref_processor shouldn't be NULL"); | |
| 7685 } | |
| 7686 | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7687 void PushAndMarkClosure::do_oop(oop* p) { PushAndMarkClosure::do_oop_work(p); } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7688 void PushAndMarkClosure::do_oop(narrowOop* p) { PushAndMarkClosure::do_oop_work(p); } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7689 |
| 0 | 7690 // Grey object rescan during second checkpoint phase -- |
| 7691 // the parallel version. | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7692 void Par_PushAndMarkClosure::do_oop(oop obj) { |
| 0 | 7693 // In the assert below, we ignore the mark word because |
| 7694 // this oop may point to an already visited object that is | |
| 7695 // on the overflow stack (in which case the mark word has | |
| 7696 // been hijacked for chaining into the overflow stack -- | |
| 7697 // if this is the last object in the overflow stack then | |
| 7698 // its mark word will be NULL). Because this object may | |
| 7699 // have been subsequently popped off the global overflow | |
| 7700 // stack, and the mark word possibly restored to the prototypical | |
| 7701 // value, by the time we get to examined this failing assert in | |
| 7702 // the debugger, is_oop_or_null(false) may subsequently start | |
| 7703 // to hold. | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7704 assert(obj->is_oop_or_null(true), |
| 0 | 7705 "expected an oop or NULL"); |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7706 HeapWord* addr = (HeapWord*)obj; |
| 0 | 7707 // Check if oop points into the CMS generation |
| 7708 // and is not marked | |
| 7709 if (_span.contains(addr) && !_bit_map->isMarked(addr)) { | |
| 7710 // a white object ... | |
| 7711 // If we manage to "claim" the object, by being the | |
| 7712 // first thread to mark it, then we push it on our | |
| 7713 // marking stack | |
| 7714 if (_bit_map->par_mark(addr)) { // ... now grey | |
| 7715 // push on work queue (grey set) | |
| 7716 bool simulate_overflow = false; | |
| 7717 NOT_PRODUCT( | |
| 7718 if (CMSMarkStackOverflowALot && | |
| 7719 _collector->par_simulate_overflow()) { | |
| 7720 // simulate a stack overflow | |
| 7721 simulate_overflow = true; | |
| 7722 } | |
| 7723 ) | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7724 if (simulate_overflow || !_work_queue->push(obj)) { |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7725 _collector->par_push_on_overflow_list(obj); |
| 0 | 7726 _collector->_par_pmc_remark_ovflw++; // imprecise OK: no need to CAS |
| 7727 } | |
| 7728 } // Else, some other thread got there first | |
| 7729 } | |
| 7730 } | |
| 7731 | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7732 void Par_PushAndMarkClosure::do_oop(oop* p) { Par_PushAndMarkClosure::do_oop_work(p); } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7733 void Par_PushAndMarkClosure::do_oop(narrowOop* p) { Par_PushAndMarkClosure::do_oop_work(p); } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
7734 |
| 941 | 7735 void PushAndMarkClosure::remember_mdo(DataLayout* v) { |
| 7736 // TBD | |
| 7737 } | |
| 7738 | |
| 7739 void Par_PushAndMarkClosure::remember_mdo(DataLayout* v) { | |
| 7740 // TBD | |
| 7741 } | |
| 7742 | |
| 0 | 7743 void CMSPrecleanRefsYieldClosure::do_yield_work() { |
| 935 | 7744 DEBUG_ONLY(RememberKlassesChecker mux(false);) |
| 0 | 7745 Mutex* bml = _collector->bitMapLock(); |
| 7746 assert_lock_strong(bml); | |
| 7747 assert(ConcurrentMarkSweepThread::cms_thread_has_cms_token(), | |
| 7748 "CMS thread should hold CMS token"); | |
| 7749 | |
| 7750 bml->unlock(); | |
| 7751 ConcurrentMarkSweepThread::desynchronize(true); | |
| 7752 | |
| 7753 ConcurrentMarkSweepThread::acknowledge_yield_request(); | |
| 7754 | |
| 7755 _collector->stopTimer(); | |
| 7756 GCPauseTimer p(_collector->size_policy()->concurrent_timer_ptr()); | |
| 7757 if (PrintCMSStatistics != 0) { | |
| 7758 _collector->incrementYields(); | |
| 7759 } | |
| 7760 _collector->icms_wait(); | |
| 7761 | |
| 7762 // See the comment in coordinator_yield() | |
| 7763 for (unsigned i = 0; i < CMSYieldSleepCount && | |
| 7764 ConcurrentMarkSweepThread::should_yield() && | |
| 7765 !CMSCollector::foregroundGCIsActive(); ++i) { | |
| 7766 os::sleep(Thread::current(), 1, false); | |
| 7767 ConcurrentMarkSweepThread::acknowledge_yield_request(); | |
| 7768 } | |
| 7769 | |
| 7770 ConcurrentMarkSweepThread::synchronize(true); | |
| 7771 bml->lock(); | |
| 7772 | |
| 7773 _collector->startTimer(); | |
| 7774 } | |
| 7775 | |
| 7776 bool CMSPrecleanRefsYieldClosure::should_return() { | |
| 7777 if (ConcurrentMarkSweepThread::should_yield()) { | |
| 7778 do_yield_work(); | |
| 7779 } | |
| 7780 return _collector->foregroundGCIsActive(); | |
| 7781 } | |
| 7782 | |
| 7783 void MarkFromDirtyCardsClosure::do_MemRegion(MemRegion mr) { | |
| 7784 assert(((size_t)mr.start())%CardTableModRefBS::card_size_in_words == 0, | |
| 7785 "mr should be aligned to start at a card boundary"); | |
| 7786 // We'd like to assert: | |
| 7787 // assert(mr.word_size()%CardTableModRefBS::card_size_in_words == 0, | |
| 7788 // "mr should be a range of cards"); | |
| 7789 // However, that would be too strong in one case -- the last | |
| 7790 // partition ends at _unallocated_block which, in general, can be | |
| 7791 // an arbitrary boundary, not necessarily card aligned. | |
| 7792 if (PrintCMSStatistics != 0) { | |
| 7793 _num_dirty_cards += | |
| 7794 mr.word_size()/CardTableModRefBS::card_size_in_words; | |
| 7795 } | |
| 7796 _space->object_iterate_mem(mr, &_scan_cl); | |
| 7797 } | |
| 7798 | |
| 7799 SweepClosure::SweepClosure(CMSCollector* collector, | |
| 7800 ConcurrentMarkSweepGeneration* g, | |
| 7801 CMSBitMap* bitMap, bool should_yield) : | |
| 7802 _collector(collector), | |
| 7803 _g(g), | |
| 7804 _sp(g->cmsSpace()), | |
| 7805 _limit(_sp->sweep_limit()), | |
| 7806 _freelistLock(_sp->freelistLock()), | |
| 7807 _bitMap(bitMap), | |
| 7808 _yield(should_yield), | |
| 7809 _inFreeRange(false), // No free range at beginning of sweep | |
| 7810 _freeRangeInFreeLists(false), // No free range at beginning of sweep | |
| 7811 _lastFreeRangeCoalesced(false), | |
| 7812 _freeFinger(g->used_region().start()) | |
| 7813 { | |
| 7814 NOT_PRODUCT( | |
| 7815 _numObjectsFreed = 0; | |
| 7816 _numWordsFreed = 0; | |
| 7817 _numObjectsLive = 0; | |
| 7818 _numWordsLive = 0; | |
| 7819 _numObjectsAlreadyFree = 0; | |
| 7820 _numWordsAlreadyFree = 0; | |
| 7821 _last_fc = NULL; | |
| 7822 | |
| 7823 _sp->initializeIndexedFreeListArrayReturnedBytes(); | |
| 7824 _sp->dictionary()->initializeDictReturnedBytes(); | |
| 7825 ) | |
| 7826 assert(_limit >= _sp->bottom() && _limit <= _sp->end(), | |
| 7827 "sweep _limit out of bounds"); | |
| 7828 if (CMSTraceSweeper) { | |
| 7829 gclog_or_tty->print("\n====================\nStarting new sweep\n"); | |
| 7830 } | |
| 7831 } | |
| 7832 | |
| 7833 // We need this destructor to reclaim any space at the end | |
| 7834 // of the space, which do_blk below may not have added back to | |
| 7835 // the free lists. [basically dealing with the "fringe effect"] | |
| 7836 SweepClosure::~SweepClosure() { | |
| 7837 assert_lock_strong(_freelistLock); | |
| 7838 // this should be treated as the end of a free run if any | |
| 7839 // The current free range should be returned to the free lists | |
| 7840 // as one coalesced chunk. | |
| 7841 if (inFreeRange()) { | |
| 7842 flushCurFreeChunk(freeFinger(), | |
| 7843 pointer_delta(_limit, freeFinger())); | |
| 7844 assert(freeFinger() < _limit, "the finger pointeth off base"); | |
| 7845 if (CMSTraceSweeper) { | |
| 7846 gclog_or_tty->print("destructor:"); | |
| 7847 gclog_or_tty->print("Sweep:put_free_blk 0x%x ("SIZE_FORMAT") " | |
| 7848 "[coalesced:"SIZE_FORMAT"]\n", | |
| 7849 freeFinger(), pointer_delta(_limit, freeFinger()), | |
| 7850 lastFreeRangeCoalesced()); | |
| 7851 } | |
| 7852 } | |
| 7853 NOT_PRODUCT( | |
| 7854 if (Verbose && PrintGC) { | |
| 7855 gclog_or_tty->print("Collected "SIZE_FORMAT" objects, " | |
| 7856 SIZE_FORMAT " bytes", | |
| 7857 _numObjectsFreed, _numWordsFreed*sizeof(HeapWord)); | |
| 7858 gclog_or_tty->print_cr("\nLive "SIZE_FORMAT" objects, " | |
| 7859 SIZE_FORMAT" bytes " | |
| 7860 "Already free "SIZE_FORMAT" objects, "SIZE_FORMAT" bytes", | |
| 7861 _numObjectsLive, _numWordsLive*sizeof(HeapWord), | |
| 7862 _numObjectsAlreadyFree, _numWordsAlreadyFree*sizeof(HeapWord)); | |
| 7863 size_t totalBytes = (_numWordsFreed + _numWordsLive + _numWordsAlreadyFree) * | |
| 7864 sizeof(HeapWord); | |
| 7865 gclog_or_tty->print_cr("Total sweep: "SIZE_FORMAT" bytes", totalBytes); | |
| 7866 | |
| 7867 if (PrintCMSStatistics && CMSVerifyReturnedBytes) { | |
| 7868 size_t indexListReturnedBytes = _sp->sumIndexedFreeListArrayReturnedBytes(); | |
| 7869 size_t dictReturnedBytes = _sp->dictionary()->sumDictReturnedBytes(); | |
| 7870 size_t returnedBytes = indexListReturnedBytes + dictReturnedBytes; | |
| 7871 gclog_or_tty->print("Returned "SIZE_FORMAT" bytes", returnedBytes); | |
| 7872 gclog_or_tty->print(" Indexed List Returned "SIZE_FORMAT" bytes", | |
| 7873 indexListReturnedBytes); | |
| 7874 gclog_or_tty->print_cr(" Dictionary Returned "SIZE_FORMAT" bytes", | |
| 7875 dictReturnedBytes); | |
| 7876 } | |
| 7877 } | |
| 7878 ) | |
| 7879 // Now, in debug mode, just null out the sweep_limit | |
| 7880 NOT_PRODUCT(_sp->clear_sweep_limit();) | |
| 7881 if (CMSTraceSweeper) { | |
| 7882 gclog_or_tty->print("end of sweep\n================\n"); | |
| 7883 } | |
| 7884 } | |
| 7885 | |
| 7886 void SweepClosure::initialize_free_range(HeapWord* freeFinger, | |
| 7887 bool freeRangeInFreeLists) { | |
| 7888 if (CMSTraceSweeper) { | |
| 7889 gclog_or_tty->print("---- Start free range 0x%x with free block [%d] (%d)\n", | |
| 7890 freeFinger, _sp->block_size(freeFinger), | |
| 7891 freeRangeInFreeLists); | |
| 7892 } | |
| 7893 assert(!inFreeRange(), "Trampling existing free range"); | |
| 7894 set_inFreeRange(true); | |
| 7895 set_lastFreeRangeCoalesced(false); | |
| 7896 | |
| 7897 set_freeFinger(freeFinger); | |
| 7898 set_freeRangeInFreeLists(freeRangeInFreeLists); | |
| 7899 if (CMSTestInFreeList) { | |
| 7900 if (freeRangeInFreeLists) { | |
| 7901 FreeChunk* fc = (FreeChunk*) freeFinger; | |
| 7902 assert(fc->isFree(), "A chunk on the free list should be free."); | |
| 7903 assert(fc->size() > 0, "Free range should have a size"); | |
| 7904 assert(_sp->verifyChunkInFreeLists(fc), "Chunk is not in free lists"); | |
| 7905 } | |
| 7906 } | |
| 7907 } | |
| 7908 | |
| 7909 // Note that the sweeper runs concurrently with mutators. Thus, | |
| 7910 // it is possible for direct allocation in this generation to happen | |
| 7911 // in the middle of the sweep. Note that the sweeper also coalesces | |
| 7912 // contiguous free blocks. Thus, unless the sweeper and the allocator | |
| 7913 // synchronize appropriately freshly allocated blocks may get swept up. | |
| 7914 // This is accomplished by the sweeper locking the free lists while | |
| 7915 // it is sweeping. Thus blocks that are determined to be free are | |
| 7916 // indeed free. There is however one additional complication: | |
| 7917 // blocks that have been allocated since the final checkpoint and | |
| 7918 // mark, will not have been marked and so would be treated as | |
| 7919 // unreachable and swept up. To prevent this, the allocator marks | |
| 7920 // the bit map when allocating during the sweep phase. This leads, | |
| 7921 // however, to a further complication -- objects may have been allocated | |
| 7922 // but not yet initialized -- in the sense that the header isn't yet | |
| 7923 // installed. The sweeper can not then determine the size of the block | |
| 7924 // in order to skip over it. To deal with this case, we use a technique | |
| 7925 // (due to Printezis) to encode such uninitialized block sizes in the | |
| 7926 // bit map. Since the bit map uses a bit per every HeapWord, but the | |
| 7927 // CMS generation has a minimum object size of 3 HeapWords, it follows | |
| 7928 // that "normal marks" won't be adjacent in the bit map (there will | |
| 7929 // always be at least two 0 bits between successive 1 bits). We make use | |
| 7930 // of these "unused" bits to represent uninitialized blocks -- the bit | |
| 7931 // corresponding to the start of the uninitialized object and the next | |
| 7932 // bit are both set. Finally, a 1 bit marks the end of the object that | |
| 7933 // started with the two consecutive 1 bits to indicate its potentially | |
| 7934 // uninitialized state. | |
| 7935 | |
| 7936 size_t SweepClosure::do_blk_careful(HeapWord* addr) { | |
| 7937 FreeChunk* fc = (FreeChunk*)addr; | |
| 7938 size_t res; | |
| 7939 | |
| 7940 // check if we are done sweepinrg | |
| 7941 if (addr == _limit) { // we have swept up to the limit, do nothing more | |
| 7942 assert(_limit >= _sp->bottom() && _limit <= _sp->end(), | |
| 7943 "sweep _limit out of bounds"); | |
| 7944 // help the closure application finish | |
| 7945 return pointer_delta(_sp->end(), _limit); | |
| 7946 } | |
| 7947 assert(addr <= _limit, "sweep invariant"); | |
| 7948 | |
| 7949 // check if we should yield | |
| 7950 do_yield_check(addr); | |
| 7951 if (fc->isFree()) { | |
| 7952 // Chunk that is already free | |
| 7953 res = fc->size(); | |
| 7954 doAlreadyFreeChunk(fc); | |
| 7955 debug_only(_sp->verifyFreeLists()); | |
| 7956 assert(res == fc->size(), "Don't expect the size to change"); | |
| 7957 NOT_PRODUCT( | |
| 7958 _numObjectsAlreadyFree++; | |
| 7959 _numWordsAlreadyFree += res; | |
| 7960 ) | |
| 7961 NOT_PRODUCT(_last_fc = fc;) | |
| 7962 } else if (!_bitMap->isMarked(addr)) { | |
| 7963 // Chunk is fresh garbage | |
| 7964 res = doGarbageChunk(fc); | |
| 7965 debug_only(_sp->verifyFreeLists()); | |
| 7966 NOT_PRODUCT( | |
| 7967 _numObjectsFreed++; | |
| 7968 _numWordsFreed += res; | |
| 7969 ) | |
| 7970 } else { | |
| 7971 // Chunk that is alive. | |
| 7972 res = doLiveChunk(fc); | |
| 7973 debug_only(_sp->verifyFreeLists()); | |
| 7974 NOT_PRODUCT( | |
| 7975 _numObjectsLive++; | |
| 7976 _numWordsLive += res; | |
| 7977 ) | |
| 7978 } | |
| 7979 return res; | |
| 7980 } | |
| 7981 | |
| 7982 // For the smart allocation, record following | |
| 7983 // split deaths - a free chunk is removed from its free list because | |
| 7984 // it is being split into two or more chunks. | |
| 7985 // split birth - a free chunk is being added to its free list because | |
| 7986 // a larger free chunk has been split and resulted in this free chunk. | |
| 7987 // coal death - a free chunk is being removed from its free list because | |
| 7988 // it is being coalesced into a large free chunk. | |
| 7989 // coal birth - a free chunk is being added to its free list because | |
| 7990 // it was created when two or more free chunks where coalesced into | |
| 7991 // this free chunk. | |
| 7992 // | |
| 7993 // These statistics are used to determine the desired number of free | |
| 7994 // chunks of a given size. The desired number is chosen to be relative | |
| 7995 // to the end of a CMS sweep. The desired number at the end of a sweep | |
| 7996 // is the | |
| 7997 // count-at-end-of-previous-sweep (an amount that was enough) | |
| 7998 // - count-at-beginning-of-current-sweep (the excess) | |
| 7999 // + split-births (gains in this size during interval) | |
| 8000 // - split-deaths (demands on this size during interval) | |
| 8001 // where the interval is from the end of one sweep to the end of the | |
| 8002 // next. | |
| 8003 // | |
| 8004 // When sweeping the sweeper maintains an accumulated chunk which is | |
| 8005 // the chunk that is made up of chunks that have been coalesced. That | |
| 8006 // will be termed the left-hand chunk. A new chunk of garbage that | |
| 8007 // is being considered for coalescing will be referred to as the | |
| 8008 // right-hand chunk. | |
| 8009 // | |
| 8010 // When making a decision on whether to coalesce a right-hand chunk with | |
| 8011 // the current left-hand chunk, the current count vs. the desired count | |
| 8012 // of the left-hand chunk is considered. Also if the right-hand chunk | |
| 8013 // is near the large chunk at the end of the heap (see | |
| 8014 // ConcurrentMarkSweepGeneration::isNearLargestChunk()), then the | |
| 8015 // left-hand chunk is coalesced. | |
| 8016 // | |
| 8017 // When making a decision about whether to split a chunk, the desired count | |
| 8018 // vs. the current count of the candidate to be split is also considered. | |
| 8019 // If the candidate is underpopulated (currently fewer chunks than desired) | |
| 8020 // a chunk of an overpopulated (currently more chunks than desired) size may | |
| 8021 // be chosen. The "hint" associated with a free list, if non-null, points | |
| 8022 // to a free list which may be overpopulated. | |
| 8023 // | |
| 8024 | |
| 8025 void SweepClosure::doAlreadyFreeChunk(FreeChunk* fc) { | |
| 8026 size_t size = fc->size(); | |
| 8027 // Chunks that cannot be coalesced are not in the | |
| 8028 // free lists. | |
| 8029 if (CMSTestInFreeList && !fc->cantCoalesce()) { | |
| 8030 assert(_sp->verifyChunkInFreeLists(fc), | |
| 8031 "free chunk should be in free lists"); | |
| 8032 } | |
| 8033 // a chunk that is already free, should not have been | |
| 8034 // marked in the bit map | |
| 8035 HeapWord* addr = (HeapWord*) fc; | |
| 8036 assert(!_bitMap->isMarked(addr), "free chunk should be unmarked"); | |
| 8037 // Verify that the bit map has no bits marked between | |
| 8038 // addr and purported end of this block. | |
| 8039 _bitMap->verifyNoOneBitsInRange(addr + 1, addr + size); | |
| 8040 | |
| 8041 // Some chunks cannot be coalesced in under any circumstances. | |
| 8042 // See the definition of cantCoalesce(). | |
| 8043 if (!fc->cantCoalesce()) { | |
| 8044 // This chunk can potentially be coalesced. | |
| 8045 if (_sp->adaptive_freelists()) { | |
| 8046 // All the work is done in | |
| 8047 doPostIsFreeOrGarbageChunk(fc, size); | |
| 8048 } else { // Not adaptive free lists | |
| 8049 // this is a free chunk that can potentially be coalesced by the sweeper; | |
| 8050 if (!inFreeRange()) { | |
| 8051 // if the next chunk is a free block that can't be coalesced | |
| 8052 // it doesn't make sense to remove this chunk from the free lists | |
| 8053 FreeChunk* nextChunk = (FreeChunk*)(addr + size); | |
| 8054 assert((HeapWord*)nextChunk <= _limit, "sweep invariant"); | |
| 8055 if ((HeapWord*)nextChunk < _limit && // there's a next chunk... | |
| 8056 nextChunk->isFree() && // which is free... | |
| 8057 nextChunk->cantCoalesce()) { // ... but cant be coalesced | |
| 8058 // nothing to do | |
| 8059 } else { | |
| 8060 // Potentially the start of a new free range: | |
| 8061 // Don't eagerly remove it from the free lists. | |
| 8062 // No need to remove it if it will just be put | |
| 8063 // back again. (Also from a pragmatic point of view | |
| 8064 // if it is a free block in a region that is beyond | |
| 8065 // any allocated blocks, an assertion will fail) | |
| 8066 // Remember the start of a free run. | |
| 8067 initialize_free_range(addr, true); | |
| 8068 // end - can coalesce with next chunk | |
| 8069 } | |
| 8070 } else { | |
| 8071 // the midst of a free range, we are coalescing | |
| 8072 debug_only(record_free_block_coalesced(fc);) | |
| 8073 if (CMSTraceSweeper) { | |
| 8074 gclog_or_tty->print(" -- pick up free block 0x%x (%d)\n", fc, size); | |
| 8075 } | |
| 8076 // remove it from the free lists | |
| 8077 _sp->removeFreeChunkFromFreeLists(fc); | |
| 8078 set_lastFreeRangeCoalesced(true); | |
| 8079 // If the chunk is being coalesced and the current free range is | |
| 8080 // in the free lists, remove the current free range so that it | |
| 8081 // will be returned to the free lists in its entirety - all | |
| 8082 // the coalesced pieces included. | |
| 8083 if (freeRangeInFreeLists()) { | |
| 8084 FreeChunk* ffc = (FreeChunk*) freeFinger(); | |
| 8085 assert(ffc->size() == pointer_delta(addr, freeFinger()), | |
| 8086 "Size of free range is inconsistent with chunk size."); | |
| 8087 if (CMSTestInFreeList) { | |
| 8088 assert(_sp->verifyChunkInFreeLists(ffc), | |
| 8089 "free range is not in free lists"); | |
| 8090 } | |
| 8091 _sp->removeFreeChunkFromFreeLists(ffc); | |
| 8092 set_freeRangeInFreeLists(false); | |
| 8093 } | |
| 8094 } | |
| 8095 } | |
| 8096 } else { | |
| 8097 // Code path common to both original and adaptive free lists. | |
| 8098 | |
| 8099 // cant coalesce with previous block; this should be treated | |
| 8100 // as the end of a free run if any | |
| 8101 if (inFreeRange()) { | |
| 8102 // we kicked some butt; time to pick up the garbage | |
| 8103 assert(freeFinger() < addr, "the finger pointeth off base"); | |
| 8104 flushCurFreeChunk(freeFinger(), pointer_delta(addr, freeFinger())); | |
| 8105 } | |
| 8106 // else, nothing to do, just continue | |
| 8107 } | |
| 8108 } | |
| 8109 | |
| 8110 size_t SweepClosure::doGarbageChunk(FreeChunk* fc) { | |
| 8111 // This is a chunk of garbage. It is not in any free list. | |
| 8112 // Add it to a free list or let it possibly be coalesced into | |
| 8113 // a larger chunk. | |
| 8114 HeapWord* addr = (HeapWord*) fc; | |
| 8115 size_t size = CompactibleFreeListSpace::adjustObjectSize(oop(addr)->size()); | |
| 8116 | |
| 8117 if (_sp->adaptive_freelists()) { | |
| 8118 // Verify that the bit map has no bits marked between | |
| 8119 // addr and purported end of just dead object. | |
| 8120 _bitMap->verifyNoOneBitsInRange(addr + 1, addr + size); | |
| 8121 | |
| 8122 doPostIsFreeOrGarbageChunk(fc, size); | |
| 8123 } else { | |
| 8124 if (!inFreeRange()) { | |
| 8125 // start of a new free range | |
| 8126 assert(size > 0, "A free range should have a size"); | |
| 8127 initialize_free_range(addr, false); | |
| 8128 | |
| 8129 } else { | |
| 8130 // this will be swept up when we hit the end of the | |
| 8131 // free range | |
| 8132 if (CMSTraceSweeper) { | |
| 8133 gclog_or_tty->print(" -- pick up garbage 0x%x (%d) \n", fc, size); | |
| 8134 } | |
| 8135 // If the chunk is being coalesced and the current free range is | |
| 8136 // in the free lists, remove the current free range so that it | |
| 8137 // will be returned to the free lists in its entirety - all | |
| 8138 // the coalesced pieces included. | |
| 8139 if (freeRangeInFreeLists()) { | |
| 8140 FreeChunk* ffc = (FreeChunk*)freeFinger(); | |
| 8141 assert(ffc->size() == pointer_delta(addr, freeFinger()), | |
| 8142 "Size of free range is inconsistent with chunk size."); | |
| 8143 if (CMSTestInFreeList) { | |
| 8144 assert(_sp->verifyChunkInFreeLists(ffc), | |
| 8145 "free range is not in free lists"); | |
| 8146 } | |
| 8147 _sp->removeFreeChunkFromFreeLists(ffc); | |
| 8148 set_freeRangeInFreeLists(false); | |
| 8149 } | |
| 8150 set_lastFreeRangeCoalesced(true); | |
| 8151 } | |
| 8152 // this will be swept up when we hit the end of the free range | |
| 8153 | |
| 8154 // Verify that the bit map has no bits marked between | |
| 8155 // addr and purported end of just dead object. | |
| 8156 _bitMap->verifyNoOneBitsInRange(addr + 1, addr + size); | |
| 8157 } | |
| 8158 return size; | |
| 8159 } | |
| 8160 | |
| 8161 size_t SweepClosure::doLiveChunk(FreeChunk* fc) { | |
| 8162 HeapWord* addr = (HeapWord*) fc; | |
| 8163 // The sweeper has just found a live object. Return any accumulated | |
| 8164 // left hand chunk to the free lists. | |
| 8165 if (inFreeRange()) { | |
| 8166 if (_sp->adaptive_freelists()) { | |
| 8167 flushCurFreeChunk(freeFinger(), | |
| 8168 pointer_delta(addr, freeFinger())); | |
| 8169 } else { // not adaptive freelists | |
| 8170 set_inFreeRange(false); | |
| 8171 // Add the free range back to the free list if it is not already | |
| 8172 // there. | |
| 8173 if (!freeRangeInFreeLists()) { | |
| 8174 assert(freeFinger() < addr, "the finger pointeth off base"); | |
| 8175 if (CMSTraceSweeper) { | |
| 8176 gclog_or_tty->print("Sweep:put_free_blk 0x%x (%d) " | |
| 8177 "[coalesced:%d]\n", | |
| 8178 freeFinger(), pointer_delta(addr, freeFinger()), | |
| 8179 lastFreeRangeCoalesced()); | |
| 8180 } | |
| 8181 _sp->addChunkAndRepairOffsetTable(freeFinger(), | |
| 8182 pointer_delta(addr, freeFinger()), lastFreeRangeCoalesced()); | |
| 8183 } | |
| 8184 } | |
| 8185 } | |
| 8186 | |
| 8187 // Common code path for original and adaptive free lists. | |
| 8188 | |
| 8189 // this object is live: we'd normally expect this to be | |
| 8190 // an oop, and like to assert the following: | |
| 8191 // assert(oop(addr)->is_oop(), "live block should be an oop"); | |
| 8192 // However, as we commented above, this may be an object whose | |
| 8193 // header hasn't yet been initialized. | |
| 8194 size_t size; | |
| 8195 assert(_bitMap->isMarked(addr), "Tautology for this control point"); | |
| 8196 if (_bitMap->isMarked(addr + 1)) { | |
| 8197 // Determine the size from the bit map, rather than trying to | |
| 8198 // compute it from the object header. | |
| 8199 HeapWord* nextOneAddr = _bitMap->getNextMarkedWordAddress(addr + 2); | |
| 8200 size = pointer_delta(nextOneAddr + 1, addr); | |
| 8201 assert(size == CompactibleFreeListSpace::adjustObjectSize(size), | |
| 8202 "alignment problem"); | |
| 8203 | |
| 8204 #ifdef DEBUG | |
| 187 | 8205 if (oop(addr)->klass_or_null() != NULL && |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
8206 ( !_collector->should_unload_classes() |
|
518
0af8b0718fc9
6692899: CMS: many vm.parallel_class_loading tests fail with assert "missing Printezis mark"
jmasa
parents:
517
diff
changeset
|
8207 || (oop(addr)->is_parsable()) && |
|
0af8b0718fc9
6692899: CMS: many vm.parallel_class_loading tests fail with assert "missing Printezis mark"
jmasa
parents:
517
diff
changeset
|
8208 oop(addr)->is_conc_safe())) { |
| 0 | 8209 // Ignore mark word because we are running concurrent with mutators |
| 8210 assert(oop(addr)->is_oop(true), "live block should be an oop"); | |
|
518
0af8b0718fc9
6692899: CMS: many vm.parallel_class_loading tests fail with assert "missing Printezis mark"
jmasa
parents:
517
diff
changeset
|
8211 // is_conc_safe is checked before performing this assertion |
|
0af8b0718fc9
6692899: CMS: many vm.parallel_class_loading tests fail with assert "missing Printezis mark"
jmasa
parents:
517
diff
changeset
|
8212 // because an object that is not is_conc_safe may yet have |
|
0af8b0718fc9
6692899: CMS: many vm.parallel_class_loading tests fail with assert "missing Printezis mark"
jmasa
parents:
517
diff
changeset
|
8213 // the return from size() correct. |
| 0 | 8214 assert(size == |
| 8215 CompactibleFreeListSpace::adjustObjectSize(oop(addr)->size()), | |
| 8216 "P-mark and computed size do not agree"); | |
| 8217 } | |
| 8218 #endif | |
| 8219 | |
| 8220 } else { | |
| 8221 // This should be an initialized object that's alive. | |
| 187 | 8222 assert(oop(addr)->klass_or_null() != NULL && |
|
94
0834225a7916
6634032: CMS: Need CMSInitiatingPermOccupancyFraction for perm, divorcing from CMSInitiatingOccupancyFraction
ysr
parents:
9
diff
changeset
|
8223 (!_collector->should_unload_classes() |
| 0 | 8224 || oop(addr)->is_parsable()), |
| 8225 "Should be an initialized object"); | |
|
518
0af8b0718fc9
6692899: CMS: many vm.parallel_class_loading tests fail with assert "missing Printezis mark"
jmasa
parents:
517
diff
changeset
|
8226 // Note that there are objects used during class redefinition |
|
0af8b0718fc9
6692899: CMS: many vm.parallel_class_loading tests fail with assert "missing Printezis mark"
jmasa
parents:
517
diff
changeset
|
8227 // (e.g., merge_cp in VM_RedefineClasses::merge_cp_and_rewrite() |
|
0af8b0718fc9
6692899: CMS: many vm.parallel_class_loading tests fail with assert "missing Printezis mark"
jmasa
parents:
517
diff
changeset
|
8228 // which are discarded with their is_conc_safe state still |
|
0af8b0718fc9
6692899: CMS: many vm.parallel_class_loading tests fail with assert "missing Printezis mark"
jmasa
parents:
517
diff
changeset
|
8229 // false. These object may be floating garbage so may be |
|
0af8b0718fc9
6692899: CMS: many vm.parallel_class_loading tests fail with assert "missing Printezis mark"
jmasa
parents:
517
diff
changeset
|
8230 // seen here. If they are floating garbage their size |
|
0af8b0718fc9
6692899: CMS: many vm.parallel_class_loading tests fail with assert "missing Printezis mark"
jmasa
parents:
517
diff
changeset
|
8231 // should be attainable from their klass. Do not that |
|
0af8b0718fc9
6692899: CMS: many vm.parallel_class_loading tests fail with assert "missing Printezis mark"
jmasa
parents:
517
diff
changeset
|
8232 // is_conc_safe() is true for oop(addr). |
| 0 | 8233 // Ignore mark word because we are running concurrent with mutators |
| 8234 assert(oop(addr)->is_oop(true), "live block should be an oop"); | |
| 8235 // Verify that the bit map has no bits marked between | |
| 8236 // addr and purported end of this block. | |
| 8237 size = CompactibleFreeListSpace::adjustObjectSize(oop(addr)->size()); | |
| 8238 assert(size >= 3, "Necessary for Printezis marks to work"); | |
| 8239 assert(!_bitMap->isMarked(addr+1), "Tautology for this control point"); | |
| 8240 DEBUG_ONLY(_bitMap->verifyNoOneBitsInRange(addr+2, addr+size);) | |
| 8241 } | |
| 8242 return size; | |
| 8243 } | |
| 8244 | |
| 8245 void SweepClosure::doPostIsFreeOrGarbageChunk(FreeChunk* fc, | |
| 8246 size_t chunkSize) { | |
| 8247 // doPostIsFreeOrGarbageChunk() should only be called in the smart allocation | |
| 8248 // scheme. | |
| 8249 bool fcInFreeLists = fc->isFree(); | |
| 8250 assert(_sp->adaptive_freelists(), "Should only be used in this case."); | |
| 8251 assert((HeapWord*)fc <= _limit, "sweep invariant"); | |
| 8252 if (CMSTestInFreeList && fcInFreeLists) { | |
| 8253 assert(_sp->verifyChunkInFreeLists(fc), | |
| 8254 "free chunk is not in free lists"); | |
| 8255 } | |
| 8256 | |
| 8257 | |
| 8258 if (CMSTraceSweeper) { | |
| 8259 gclog_or_tty->print_cr(" -- pick up another chunk at 0x%x (%d)", fc, chunkSize); | |
| 8260 } | |
| 8261 | |
| 8262 HeapWord* addr = (HeapWord*) fc; | |
| 8263 | |
| 8264 bool coalesce; | |
| 8265 size_t left = pointer_delta(addr, freeFinger()); | |
| 8266 size_t right = chunkSize; | |
| 8267 switch (FLSCoalescePolicy) { | |
| 8268 // numeric value forms a coalition aggressiveness metric | |
| 8269 case 0: { // never coalesce | |
| 8270 coalesce = false; | |
| 8271 break; | |
| 8272 } | |
| 8273 case 1: { // coalesce if left & right chunks on overpopulated lists | |
| 8274 coalesce = _sp->coalOverPopulated(left) && | |
| 8275 _sp->coalOverPopulated(right); | |
| 8276 break; | |
| 8277 } | |
| 8278 case 2: { // coalesce if left chunk on overpopulated list (default) | |
| 8279 coalesce = _sp->coalOverPopulated(left); | |
| 8280 break; | |
| 8281 } | |
| 8282 case 3: { // coalesce if left OR right chunk on overpopulated list | |
| 8283 coalesce = _sp->coalOverPopulated(left) || | |
| 8284 _sp->coalOverPopulated(right); | |
| 8285 break; | |
| 8286 } | |
| 8287 case 4: { // always coalesce | |
| 8288 coalesce = true; | |
| 8289 break; | |
| 8290 } | |
| 8291 default: | |
| 8292 ShouldNotReachHere(); | |
| 8293 } | |
| 8294 | |
| 8295 // Should the current free range be coalesced? | |
| 8296 // If the chunk is in a free range and either we decided to coalesce above | |
| 8297 // or the chunk is near the large block at the end of the heap | |
| 8298 // (isNearLargestChunk() returns true), then coalesce this chunk. | |
| 8299 bool doCoalesce = inFreeRange() && | |
| 8300 (coalesce || _g->isNearLargestChunk((HeapWord*)fc)); | |
| 8301 if (doCoalesce) { | |
| 8302 // Coalesce the current free range on the left with the new | |
| 8303 // chunk on the right. If either is on a free list, | |
| 8304 // it must be removed from the list and stashed in the closure. | |
| 8305 if (freeRangeInFreeLists()) { | |
| 8306 FreeChunk* ffc = (FreeChunk*)freeFinger(); | |
| 8307 assert(ffc->size() == pointer_delta(addr, freeFinger()), | |
| 8308 "Size of free range is inconsistent with chunk size."); | |
| 8309 if (CMSTestInFreeList) { | |
| 8310 assert(_sp->verifyChunkInFreeLists(ffc), | |
| 8311 "Chunk is not in free lists"); | |
| 8312 } | |
| 8313 _sp->coalDeath(ffc->size()); | |
| 8314 _sp->removeFreeChunkFromFreeLists(ffc); | |
| 8315 set_freeRangeInFreeLists(false); | |
| 8316 } | |
| 8317 if (fcInFreeLists) { | |
| 8318 _sp->coalDeath(chunkSize); | |
| 8319 assert(fc->size() == chunkSize, | |
| 8320 "The chunk has the wrong size or is not in the free lists"); | |
| 8321 _sp->removeFreeChunkFromFreeLists(fc); | |
| 8322 } | |
| 8323 set_lastFreeRangeCoalesced(true); | |
| 8324 } else { // not in a free range and/or should not coalesce | |
| 8325 // Return the current free range and start a new one. | |
| 8326 if (inFreeRange()) { | |
| 8327 // In a free range but cannot coalesce with the right hand chunk. | |
| 8328 // Put the current free range into the free lists. | |
| 8329 flushCurFreeChunk(freeFinger(), | |
| 8330 pointer_delta(addr, freeFinger())); | |
| 8331 } | |
| 8332 // Set up for new free range. Pass along whether the right hand | |
| 8333 // chunk is in the free lists. | |
| 8334 initialize_free_range((HeapWord*)fc, fcInFreeLists); | |
| 8335 } | |
| 8336 } | |
| 8337 void SweepClosure::flushCurFreeChunk(HeapWord* chunk, size_t size) { | |
| 8338 assert(inFreeRange(), "Should only be called if currently in a free range."); | |
| 8339 assert(size > 0, | |
| 8340 "A zero sized chunk cannot be added to the free lists."); | |
| 8341 if (!freeRangeInFreeLists()) { | |
| 8342 if(CMSTestInFreeList) { | |
| 8343 FreeChunk* fc = (FreeChunk*) chunk; | |
| 8344 fc->setSize(size); | |
| 8345 assert(!_sp->verifyChunkInFreeLists(fc), | |
| 8346 "chunk should not be in free lists yet"); | |
| 8347 } | |
| 8348 if (CMSTraceSweeper) { | |
| 8349 gclog_or_tty->print_cr(" -- add free block 0x%x (%d) to free lists", | |
| 8350 chunk, size); | |
| 8351 } | |
| 8352 // A new free range is going to be starting. The current | |
| 8353 // free range has not been added to the free lists yet or | |
| 8354 // was removed so add it back. | |
| 8355 // If the current free range was coalesced, then the death | |
| 8356 // of the free range was recorded. Record a birth now. | |
| 8357 if (lastFreeRangeCoalesced()) { | |
| 8358 _sp->coalBirth(size); | |
| 8359 } | |
| 8360 _sp->addChunkAndRepairOffsetTable(chunk, size, | |
| 8361 lastFreeRangeCoalesced()); | |
| 8362 } | |
| 8363 set_inFreeRange(false); | |
| 8364 set_freeRangeInFreeLists(false); | |
| 8365 } | |
| 8366 | |
| 8367 // We take a break if we've been at this for a while, | |
| 8368 // so as to avoid monopolizing the locks involved. | |
| 8369 void SweepClosure::do_yield_work(HeapWord* addr) { | |
| 8370 // Return current free chunk being used for coalescing (if any) | |
| 8371 // to the appropriate freelist. After yielding, the next | |
| 8372 // free block encountered will start a coalescing range of | |
| 8373 // free blocks. If the next free block is adjacent to the | |
| 8374 // chunk just flushed, they will need to wait for the next | |
| 8375 // sweep to be coalesced. | |
| 8376 if (inFreeRange()) { | |
| 8377 flushCurFreeChunk(freeFinger(), pointer_delta(addr, freeFinger())); | |
| 8378 } | |
| 8379 | |
| 8380 // First give up the locks, then yield, then re-lock. | |
| 8381 // We should probably use a constructor/destructor idiom to | |
| 8382 // do this unlock/lock or modify the MutexUnlocker class to | |
| 8383 // serve our purpose. XXX | |
| 8384 assert_lock_strong(_bitMap->lock()); | |
| 8385 assert_lock_strong(_freelistLock); | |
| 8386 assert(ConcurrentMarkSweepThread::cms_thread_has_cms_token(), | |
| 8387 "CMS thread should hold CMS token"); | |
| 8388 _bitMap->lock()->unlock(); | |
| 8389 _freelistLock->unlock(); | |
| 8390 ConcurrentMarkSweepThread::desynchronize(true); | |
| 8391 ConcurrentMarkSweepThread::acknowledge_yield_request(); | |
| 8392 _collector->stopTimer(); | |
| 8393 GCPauseTimer p(_collector->size_policy()->concurrent_timer_ptr()); | |
| 8394 if (PrintCMSStatistics != 0) { | |
| 8395 _collector->incrementYields(); | |
| 8396 } | |
| 8397 _collector->icms_wait(); | |
| 8398 | |
| 8399 // See the comment in coordinator_yield() | |
| 8400 for (unsigned i = 0; i < CMSYieldSleepCount && | |
| 8401 ConcurrentMarkSweepThread::should_yield() && | |
| 8402 !CMSCollector::foregroundGCIsActive(); ++i) { | |
| 8403 os::sleep(Thread::current(), 1, false); | |
| 8404 ConcurrentMarkSweepThread::acknowledge_yield_request(); | |
| 8405 } | |
| 8406 | |
| 8407 ConcurrentMarkSweepThread::synchronize(true); | |
| 8408 _freelistLock->lock(); | |
| 8409 _bitMap->lock()->lock_without_safepoint_check(); | |
| 8410 _collector->startTimer(); | |
| 8411 } | |
| 8412 | |
| 8413 #ifndef PRODUCT | |
| 8414 // This is actually very useful in a product build if it can | |
| 8415 // be called from the debugger. Compile it into the product | |
| 8416 // as needed. | |
| 8417 bool debug_verifyChunkInFreeLists(FreeChunk* fc) { | |
| 8418 return debug_cms_space->verifyChunkInFreeLists(fc); | |
| 8419 } | |
| 8420 | |
| 8421 void SweepClosure::record_free_block_coalesced(FreeChunk* fc) const { | |
| 8422 if (CMSTraceSweeper) { | |
| 8423 gclog_or_tty->print("Sweep:coal_free_blk 0x%x (%d)\n", fc, fc->size()); | |
| 8424 } | |
| 8425 } | |
| 8426 #endif | |
| 8427 | |
| 8428 // CMSIsAliveClosure | |
| 8429 bool CMSIsAliveClosure::do_object_b(oop obj) { | |
| 8430 HeapWord* addr = (HeapWord*)obj; | |
| 8431 return addr != NULL && | |
| 8432 (!_span.contains(addr) || _bit_map->isMarked(addr)); | |
| 8433 } | |
| 8434 | |
| 935 | 8435 CMSKeepAliveClosure::CMSKeepAliveClosure( CMSCollector* collector, |
| 8436 MemRegion span, | |
| 8437 CMSBitMap* bit_map, CMSMarkStack* mark_stack, | |
| 8438 CMSMarkStack* revisit_stack, bool cpc): | |
| 8439 KlassRememberingOopClosure(collector, NULL, revisit_stack), | |
| 8440 _span(span), | |
| 8441 _bit_map(bit_map), | |
| 8442 _mark_stack(mark_stack), | |
| 8443 _concurrent_precleaning(cpc) { | |
| 8444 assert(!_span.is_empty(), "Empty span could spell trouble"); | |
| 8445 } | |
| 8446 | |
| 8447 | |
| 0 | 8448 // CMSKeepAliveClosure: the serial version |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
8449 void CMSKeepAliveClosure::do_oop(oop obj) { |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
8450 HeapWord* addr = (HeapWord*)obj; |
| 0 | 8451 if (_span.contains(addr) && |
| 8452 !_bit_map->isMarked(addr)) { | |
| 8453 _bit_map->mark(addr); | |
| 8454 bool simulate_overflow = false; | |
| 8455 NOT_PRODUCT( | |
| 8456 if (CMSMarkStackOverflowALot && | |
| 8457 _collector->simulate_overflow()) { | |
| 8458 // simulate a stack overflow | |
| 8459 simulate_overflow = true; | |
| 8460 } | |
| 8461 ) | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
8462 if (simulate_overflow || !_mark_stack->push(obj)) { |
|
452
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8463 if (_concurrent_precleaning) { |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8464 // We dirty the overflown object and let the remark |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8465 // phase deal with it. |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8466 assert(_collector->overflow_list_is_empty(), "Error"); |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8467 // In the case of object arrays, we need to dirty all of |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8468 // the cards that the object spans. No locking or atomics |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8469 // are needed since no one else can be mutating the mod union |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8470 // table. |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8471 if (obj->is_objArray()) { |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8472 size_t sz = obj->size(); |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8473 HeapWord* end_card_addr = |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8474 (HeapWord*)round_to((intptr_t)(addr+sz), CardTableModRefBS::card_size); |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8475 MemRegion redirty_range = MemRegion(addr, end_card_addr); |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8476 assert(!redirty_range.is_empty(), "Arithmetical tautology"); |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8477 _collector->_modUnionTable.mark_range(redirty_range); |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8478 } else { |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8479 _collector->_modUnionTable.mark(addr); |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8480 } |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8481 _collector->_ser_kac_preclean_ovflw++; |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8482 } else { |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8483 _collector->push_on_overflow_list(obj); |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8484 _collector->_ser_kac_ovflw++; |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8485 } |
| 0 | 8486 } |
| 8487 } | |
| 8488 } | |
| 8489 | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
8490 void CMSKeepAliveClosure::do_oop(oop* p) { CMSKeepAliveClosure::do_oop_work(p); } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
8491 void CMSKeepAliveClosure::do_oop(narrowOop* p) { CMSKeepAliveClosure::do_oop_work(p); } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
8492 |
| 0 | 8493 // CMSParKeepAliveClosure: a parallel version of the above. |
| 8494 // The work queues are private to each closure (thread), | |
| 8495 // but (may be) available for stealing by other threads. | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
8496 void CMSParKeepAliveClosure::do_oop(oop obj) { |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
8497 HeapWord* addr = (HeapWord*)obj; |
| 0 | 8498 if (_span.contains(addr) && |
| 8499 !_bit_map->isMarked(addr)) { | |
| 8500 // In general, during recursive tracing, several threads | |
| 8501 // may be concurrently getting here; the first one to | |
| 8502 // "tag" it, claims it. | |
| 8503 if (_bit_map->par_mark(addr)) { | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
8504 bool res = _work_queue->push(obj); |
| 0 | 8505 assert(res, "Low water mark should be much less than capacity"); |
| 8506 // Do a recursive trim in the hope that this will keep | |
| 8507 // stack usage lower, but leave some oops for potential stealers | |
| 8508 trim_queue(_low_water_mark); | |
| 8509 } // Else, another thread got there first | |
| 8510 } | |
| 8511 } | |
| 8512 | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
8513 void CMSParKeepAliveClosure::do_oop(oop* p) { CMSParKeepAliveClosure::do_oop_work(p); } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
8514 void CMSParKeepAliveClosure::do_oop(narrowOop* p) { CMSParKeepAliveClosure::do_oop_work(p); } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
8515 |
| 0 | 8516 void CMSParKeepAliveClosure::trim_queue(uint max) { |
| 8517 while (_work_queue->size() > max) { | |
| 8518 oop new_oop; | |
| 8519 if (_work_queue->pop_local(new_oop)) { | |
| 8520 assert(new_oop != NULL && new_oop->is_oop(), "Expected an oop"); | |
| 8521 assert(_bit_map->isMarked((HeapWord*)new_oop), | |
| 8522 "no white objects on this stack!"); | |
| 8523 assert(_span.contains((HeapWord*)new_oop), "Out of bounds oop"); | |
| 8524 // iterate over the oops in this oop, marking and pushing | |
| 8525 // the ones in CMS heap (i.e. in _span). | |
| 8526 new_oop->oop_iterate(&_mark_and_push); | |
| 8527 } | |
| 8528 } | |
| 8529 } | |
| 8530 | |
| 935 | 8531 CMSInnerParMarkAndPushClosure::CMSInnerParMarkAndPushClosure( |
| 8532 CMSCollector* collector, | |
| 8533 MemRegion span, CMSBitMap* bit_map, | |
| 8534 CMSMarkStack* revisit_stack, | |
| 8535 OopTaskQueue* work_queue): | |
| 8536 Par_KlassRememberingOopClosure(collector, NULL, revisit_stack), | |
| 8537 _span(span), | |
| 8538 _bit_map(bit_map), | |
| 8539 _work_queue(work_queue) { } | |
| 8540 | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
8541 void CMSInnerParMarkAndPushClosure::do_oop(oop obj) { |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
8542 HeapWord* addr = (HeapWord*)obj; |
| 0 | 8543 if (_span.contains(addr) && |
| 8544 !_bit_map->isMarked(addr)) { | |
| 8545 if (_bit_map->par_mark(addr)) { | |
| 8546 bool simulate_overflow = false; | |
| 8547 NOT_PRODUCT( | |
| 8548 if (CMSMarkStackOverflowALot && | |
| 8549 _collector->par_simulate_overflow()) { | |
| 8550 // simulate a stack overflow | |
| 8551 simulate_overflow = true; | |
| 8552 } | |
| 8553 ) | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
8554 if (simulate_overflow || !_work_queue->push(obj)) { |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
8555 _collector->par_push_on_overflow_list(obj); |
| 0 | 8556 _collector->_par_kac_ovflw++; |
| 8557 } | |
| 8558 } // Else another thread got there already | |
| 8559 } | |
| 8560 } | |
| 8561 | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
8562 void CMSInnerParMarkAndPushClosure::do_oop(oop* p) { CMSInnerParMarkAndPushClosure::do_oop_work(p); } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
8563 void CMSInnerParMarkAndPushClosure::do_oop(narrowOop* p) { CMSInnerParMarkAndPushClosure::do_oop_work(p); } |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
8564 |
| 0 | 8565 ////////////////////////////////////////////////////////////////// |
| 8566 // CMSExpansionCause ///////////////////////////// | |
| 8567 ////////////////////////////////////////////////////////////////// | |
| 8568 const char* CMSExpansionCause::to_string(CMSExpansionCause::Cause cause) { | |
| 8569 switch (cause) { | |
| 8570 case _no_expansion: | |
| 8571 return "No expansion"; | |
| 8572 case _satisfy_free_ratio: | |
| 8573 return "Free ratio"; | |
| 8574 case _satisfy_promotion: | |
| 8575 return "Satisfy promotion"; | |
| 8576 case _satisfy_allocation: | |
| 8577 return "allocation"; | |
| 8578 case _allocate_par_lab: | |
| 8579 return "Par LAB"; | |
| 8580 case _allocate_par_spooling_space: | |
| 8581 return "Par Spooling Space"; | |
| 8582 case _adaptive_size_policy: | |
| 8583 return "Ergonomics"; | |
| 8584 default: | |
| 8585 return "unknown"; | |
| 8586 } | |
| 8587 } | |
| 8588 | |
| 8589 void CMSDrainMarkingStackClosure::do_void() { | |
| 8590 // the max number to take from overflow list at a time | |
| 8591 const size_t num = _mark_stack->capacity()/4; | |
|
452
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8592 assert(!_concurrent_precleaning || _collector->overflow_list_is_empty(), |
|
00b023ae2d78
6722113: CMS: Incorrect overflow handling during precleaning of Reference lists
ysr
parents:
360
diff
changeset
|
8593 "Overflow list should be NULL during concurrent phases"); |
| 0 | 8594 while (!_mark_stack->isEmpty() || |
| 8595 // if stack is empty, check the overflow list | |
| 8596 _collector->take_from_overflow_list(num, _mark_stack)) { | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
8597 oop obj = _mark_stack->pop(); |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
8598 HeapWord* addr = (HeapWord*)obj; |
| 0 | 8599 assert(_span.contains(addr), "Should be within span"); |
| 8600 assert(_bit_map->isMarked(addr), "Should be marked"); | |
|
113
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
8601 assert(obj->is_oop(), "Should be an oop"); |
|
ba764ed4b6f2
6420645: Create a vm that uses compressed oops for up to 32gb heapsizes
coleenp
parents:
94
diff
changeset
|
8602 obj->oop_iterate(_keep_alive); |
| 0 | 8603 } |
| 8604 } | |
| 8605 | |
| 8606 void CMSParDrainMarkingStackClosure::do_void() { | |
| 8607 // drain queue | |
| 8608 trim_queue(0); | |
| 8609 } | |
| 8610 | |
| 8611 // Trim our work_queue so its length is below max at return | |
| 8612 void CMSParDrainMarkingStackClosure::trim_queue(uint max) { | |
| 8613 while (_work_queue->size() > max) { | |
| 8614 oop new_oop; | |
| 8615 if (_work_queue->pop_local(new_oop)) { | |
| 8616 assert(new_oop->is_oop(), "Expected an oop"); | |
| 8617 assert(_bit_map->isMarked((HeapWord*)new_oop), | |
| 8618 "no white objects on this stack!"); | |
| 8619 assert(_span.contains((HeapWord*)new_oop), "Out of bounds oop"); | |
| 8620 // iterate over the oops in this oop, marking and pushing | |
| 8621 // the ones in CMS heap (i.e. in _span). | |
| 8622 new_oop->oop_iterate(&_mark_and_push); | |
| 8623 } | |
| 8624 } | |
| 8625 } | |
| 8626 | |
| 8627 //////////////////////////////////////////////////////////////////// | |
| 8628 // Support for Marking Stack Overflow list handling and related code | |
| 8629 //////////////////////////////////////////////////////////////////// | |
| 8630 // Much of the following code is similar in shape and spirit to the | |
| 8631 // code used in ParNewGC. We should try and share that code | |
| 8632 // as much as possible in the future. | |
| 8633 | |
| 8634 #ifndef PRODUCT | |
| 8635 // Debugging support for CMSStackOverflowALot | |
| 8636 | |
| 8637 // It's OK to call this multi-threaded; the worst thing | |
| 8638 // that can happen is that we'll get a bunch of closely | |
| 8639 // spaced simulated oveflows, but that's OK, in fact | |
| 8640 // probably good as it would exercise the overflow code | |
| 8641 // under contention. | |
| 8642 bool CMSCollector::simulate_overflow() { | |
| 8643 if (_overflow_counter-- <= 0) { // just being defensive | |
| 8644 _overflow_counter = CMSMarkStackOverflowInterval; | |
| 8645 return true; | |
| 8646 } else { | |
| 8647 return false; | |
| 8648 } | |
| 8649 } | |
| 8650 | |
| 8651 bool CMSCollector::par_simulate_overflow() { | |
| 8652 return simulate_overflow(); | |
| 8653 } | |
| 8654 #endif | |
| 8655 | |
| 8656 // Single-threaded | |
| 8657 bool CMSCollector::take_from_overflow_list(size_t num, CMSMarkStack* stack) { | |
| 8658 assert(stack->isEmpty(), "Expected precondition"); | |
| 8659 assert(stack->capacity() > num, "Shouldn't bite more than can chew"); | |
| 8660 size_t i = num; | |
| 8661 oop cur = _overflow_list; | |
| 8662 const markOop proto = markOopDesc::prototype(); | |
| 534 | 8663 NOT_PRODUCT(ssize_t n = 0;) |
| 0 | 8664 for (oop next; i > 0 && cur != NULL; cur = next, i--) { |
| 8665 next = oop(cur->mark()); | |
| 8666 cur->set_mark(proto); // until proven otherwise | |
| 8667 assert(cur->is_oop(), "Should be an oop"); | |
| 8668 bool res = stack->push(cur); | |
| 8669 assert(res, "Bit off more than can chew?"); | |
| 8670 NOT_PRODUCT(n++;) | |
| 8671 } | |
| 8672 _overflow_list = cur; | |
| 8673 #ifndef PRODUCT | |
| 8674 assert(_num_par_pushes >= n, "Too many pops?"); | |
| 8675 _num_par_pushes -=n; | |
| 8676 #endif | |
| 8677 return !stack->isEmpty(); | |
| 8678 } | |
| 8679 | |
| 534 | 8680 #define BUSY (oop(0x1aff1aff)) |
| 8681 // (MT-safe) Get a prefix of at most "num" from the list. | |
| 8682 // The overflow list is chained through the mark word of | |
| 8683 // each object in the list. We fetch the entire list, | |
| 8684 // break off a prefix of the right size and return the | |
| 8685 // remainder. If other threads try to take objects from | |
| 8686 // the overflow list at that time, they will wait for | |
| 8687 // some time to see if data becomes available. If (and | |
| 8688 // only if) another thread places one or more object(s) | |
| 8689 // on the global list before we have returned the suffix | |
| 8690 // to the global list, we will walk down our local list | |
| 8691 // to find its end and append the global list to | |
| 8692 // our suffix before returning it. This suffix walk can | |
| 8693 // prove to be expensive (quadratic in the amount of traffic) | |
| 8694 // when there are many objects in the overflow list and | |
| 8695 // there is much producer-consumer contention on the list. | |
| 8696 // *NOTE*: The overflow list manipulation code here and | |
| 8697 // in ParNewGeneration:: are very similar in shape, | |
| 8698 // except that in the ParNew case we use the old (from/eden) | |
| 8699 // copy of the object to thread the list via its klass word. | |
| 8700 // Because of the common code, if you make any changes in | |
| 8701 // the code below, please check the ParNew version to see if | |
| 8702 // similar changes might be needed. | |
| 8703 // CR 6797058 has been filed to consolidate the common code. | |
| 0 | 8704 bool CMSCollector::par_take_from_overflow_list(size_t num, |
| 8705 OopTaskQueue* work_q) { | |
| 534 | 8706 assert(work_q->size() == 0, "First empty local work queue"); |
| 0 | 8707 assert(num < work_q->max_elems(), "Can't bite more than we can chew"); |
| 8708 if (_overflow_list == NULL) { | |
| 8709 return false; | |
| 8710 } | |
| 8711 // Grab the entire list; we'll put back a suffix | |
| 534 | 8712 oop prefix = (oop)Atomic::xchg_ptr(BUSY, &_overflow_list); |
| 8713 Thread* tid = Thread::current(); | |
| 8714 size_t CMSOverflowSpinCount = (size_t)ParallelGCThreads; | |
| 8715 size_t sleep_time_millis = MAX2((size_t)1, num/100); | |
| 8716 // If the list is busy, we spin for a short while, | |
| 8717 // sleeping between attempts to get the list. | |
| 8718 for (size_t spin = 0; prefix == BUSY && spin < CMSOverflowSpinCount; spin++) { | |
| 8719 os::sleep(tid, sleep_time_millis, false); | |
| 8720 if (_overflow_list == NULL) { | |
| 8721 // Nothing left to take | |
| 8722 return false; | |
| 8723 } else if (_overflow_list != BUSY) { | |
| 8724 // Try and grab the prefix | |
| 8725 prefix = (oop)Atomic::xchg_ptr(BUSY, &_overflow_list); | |
| 8726 } | |
| 8727 } | |
| 8728 // If the list was found to be empty, or we spun long | |
| 8729 // enough, we give up and return empty-handed. If we leave | |
| 8730 // the list in the BUSY state below, it must be the case that | |
| 8731 // some other thread holds the overflow list and will set it | |
| 8732 // to a non-BUSY state in the future. | |
| 8733 if (prefix == NULL || prefix == BUSY) { | |
| 8734 // Nothing to take or waited long enough | |
| 8735 if (prefix == NULL) { | |
| 8736 // Write back the NULL in case we overwrote it with BUSY above | |
| 8737 // and it is still the same value. | |
| 8738 (void) Atomic::cmpxchg_ptr(NULL, &_overflow_list, BUSY); | |
| 8739 } | |
| 8740 return false; | |
| 8741 } | |
| 8742 assert(prefix != NULL && prefix != BUSY, "Error"); | |
| 0 | 8743 size_t i = num; |
| 8744 oop cur = prefix; | |
| 534 | 8745 // Walk down the first "num" objects, unless we reach the end. |
| 0 | 8746 for (; i > 1 && cur->mark() != NULL; cur = oop(cur->mark()), i--); |
| 534 | 8747 if (cur->mark() == NULL) { |
| 8748 // We have "num" or fewer elements in the list, so there | |
| 8749 // is nothing to return to the global list. | |
| 8750 // Write back the NULL in lieu of the BUSY we wrote | |
| 8751 // above, if it is still the same value. | |
| 8752 if (_overflow_list == BUSY) { | |
| 8753 (void) Atomic::cmpxchg_ptr(NULL, &_overflow_list, BUSY); | |
| 8754 } | |
| 8755 } else { | |
| 8756 // Chop off the suffix and rerturn it to the global list. | |
| 8757 assert(cur->mark() != BUSY, "Error"); | |
| 0 | 8758 oop suffix_head = cur->mark(); // suffix will be put back on global list |
| 8759 cur->set_mark(NULL); // break off suffix | |
| 534 | 8760 // It's possible that the list is still in the empty(busy) state |
| 8761 // we left it in a short while ago; in that case we may be | |
| 8762 // able to place back the suffix without incurring the cost | |
| 8763 // of a walk down the list. | |
| 0 | 8764 oop observed_overflow_list = _overflow_list; |
| 534 | 8765 oop cur_overflow_list = observed_overflow_list; |
| 8766 bool attached = false; | |
| 8767 while (observed_overflow_list == BUSY || observed_overflow_list == NULL) { | |
| 0 | 8768 observed_overflow_list = |
| 534 | 8769 (oop) Atomic::cmpxchg_ptr(suffix_head, &_overflow_list, cur_overflow_list); |
| 8770 if (cur_overflow_list == observed_overflow_list) { | |
| 8771 attached = true; | |
| 8772 break; | |
| 8773 } else cur_overflow_list = observed_overflow_list; | |
| 8774 } | |
| 8775 if (!attached) { | |
| 8776 // Too bad, someone else sneaked in (at least) an element; we'll need | |
| 8777 // to do a splice. Find tail of suffix so we can prepend suffix to global | |
| 8778 // list. | |
| 8779 for (cur = suffix_head; cur->mark() != NULL; cur = (oop)(cur->mark())); | |
| 8780 oop suffix_tail = cur; | |
| 8781 assert(suffix_tail != NULL && suffix_tail->mark() == NULL, | |
| 8782 "Tautology"); | |
| 8783 observed_overflow_list = _overflow_list; | |
| 8784 do { | |
| 8785 cur_overflow_list = observed_overflow_list; | |
| 8786 if (cur_overflow_list != BUSY) { | |
| 8787 // Do the splice ... | |
| 8788 suffix_tail->set_mark(markOop(cur_overflow_list)); | |
| 8789 } else { // cur_overflow_list == BUSY | |
| 8790 suffix_tail->set_mark(NULL); | |
| 8791 } | |
| 8792 // ... and try to place spliced list back on overflow_list ... | |
| 8793 observed_overflow_list = | |
| 8794 (oop) Atomic::cmpxchg_ptr(suffix_head, &_overflow_list, cur_overflow_list); | |
| 8795 } while (cur_overflow_list != observed_overflow_list); | |
| 8796 // ... until we have succeeded in doing so. | |
| 8797 } | |
| 0 | 8798 } |
| 8799 | |
| 8800 // Push the prefix elements on work_q | |
| 8801 assert(prefix != NULL, "control point invariant"); | |
| 8802 const markOop proto = markOopDesc::prototype(); | |
| 8803 oop next; | |
| 534 | 8804 NOT_PRODUCT(ssize_t n = 0;) |
| 0 | 8805 for (cur = prefix; cur != NULL; cur = next) { |
| 8806 next = oop(cur->mark()); | |
| 8807 cur->set_mark(proto); // until proven otherwise | |
| 8808 assert(cur->is_oop(), "Should be an oop"); | |
| 8809 bool res = work_q->push(cur); | |
| 8810 assert(res, "Bit off more than we can chew?"); | |
| 8811 NOT_PRODUCT(n++;) | |
| 8812 } | |
| 8813 #ifndef PRODUCT | |
| 8814 assert(_num_par_pushes >= n, "Too many pops?"); | |
| 8815 Atomic::add_ptr(-(intptr_t)n, &_num_par_pushes); | |
| 8816 #endif | |
| 8817 return true; | |
| 8818 } | |
| 8819 | |
| 8820 // Single-threaded | |
| 8821 void CMSCollector::push_on_overflow_list(oop p) { | |
| 8822 NOT_PRODUCT(_num_par_pushes++;) | |
| 8823 assert(p->is_oop(), "Not an oop"); | |
| 8824 preserve_mark_if_necessary(p); | |
| 8825 p->set_mark((markOop)_overflow_list); | |
| 8826 _overflow_list = p; | |
| 8827 } | |
| 8828 | |
| 8829 // Multi-threaded; use CAS to prepend to overflow list | |
| 8830 void CMSCollector::par_push_on_overflow_list(oop p) { | |
| 8831 NOT_PRODUCT(Atomic::inc_ptr(&_num_par_pushes);) | |
| 8832 assert(p->is_oop(), "Not an oop"); | |
| 8833 par_preserve_mark_if_necessary(p); | |
| 8834 oop observed_overflow_list = _overflow_list; | |
| 8835 oop cur_overflow_list; | |
| 8836 do { | |
| 8837 cur_overflow_list = observed_overflow_list; | |
| 534 | 8838 if (cur_overflow_list != BUSY) { |
| 8839 p->set_mark(markOop(cur_overflow_list)); | |
| 8840 } else { | |
| 8841 p->set_mark(NULL); | |
| 8842 } | |
| 0 | 8843 observed_overflow_list = |
| 8844 (oop) Atomic::cmpxchg_ptr(p, &_overflow_list, cur_overflow_list); | |
| 8845 } while (cur_overflow_list != observed_overflow_list); | |
| 8846 } | |
| 534 | 8847 #undef BUSY |
| 0 | 8848 |
| 8849 // Single threaded | |
| 8850 // General Note on GrowableArray: pushes may silently fail | |
| 8851 // because we are (temporarily) out of C-heap for expanding | |
| 8852 // the stack. The problem is quite ubiquitous and affects | |
| 8853 // a lot of code in the JVM. The prudent thing for GrowableArray | |
| 8854 // to do (for now) is to exit with an error. However, that may | |
| 8855 // be too draconian in some cases because the caller may be | |
| 534 | 8856 // able to recover without much harm. For such cases, we |
| 0 | 8857 // should probably introduce a "soft_push" method which returns |
| 8858 // an indication of success or failure with the assumption that | |
| 8859 // the caller may be able to recover from a failure; code in | |
| 8860 // the VM can then be changed, incrementally, to deal with such | |
| 8861 // failures where possible, thus, incrementally hardening the VM | |
| 8862 // in such low resource situations. | |
| 8863 void CMSCollector::preserve_mark_work(oop p, markOop m) { | |
| 8864 if (_preserved_oop_stack == NULL) { | |
| 8865 assert(_preserved_mark_stack == NULL, | |
| 8866 "bijection with preserved_oop_stack"); | |
| 8867 // Allocate the stacks | |
| 8868 _preserved_oop_stack = new (ResourceObj::C_HEAP) | |
| 8869 GrowableArray<oop>(PreserveMarkStackSize, true); | |
| 8870 _preserved_mark_stack = new (ResourceObj::C_HEAP) | |
| 8871 GrowableArray<markOop>(PreserveMarkStackSize, true); | |
| 8872 if (_preserved_oop_stack == NULL || _preserved_mark_stack == NULL) { | |
| 8873 vm_exit_out_of_memory(2* PreserveMarkStackSize * sizeof(oop) /* punt */, | |
| 8874 "Preserved Mark/Oop Stack for CMS (C-heap)"); | |
| 8875 } | |
| 8876 } | |
| 8877 _preserved_oop_stack->push(p); | |
| 8878 _preserved_mark_stack->push(m); | |
| 8879 assert(m == p->mark(), "Mark word changed"); | |
| 8880 assert(_preserved_oop_stack->length() == _preserved_mark_stack->length(), | |
| 8881 "bijection"); | |
| 8882 } | |
| 8883 | |
| 8884 // Single threaded | |
| 8885 void CMSCollector::preserve_mark_if_necessary(oop p) { | |
| 8886 markOop m = p->mark(); | |
| 8887 if (m->must_be_preserved(p)) { | |
| 8888 preserve_mark_work(p, m); | |
| 8889 } | |
| 8890 } | |
| 8891 | |
| 8892 void CMSCollector::par_preserve_mark_if_necessary(oop p) { | |
| 8893 markOop m = p->mark(); | |
| 8894 if (m->must_be_preserved(p)) { | |
| 8895 MutexLockerEx x(ParGCRareEvent_lock, Mutex::_no_safepoint_check_flag); | |
| 8896 // Even though we read the mark word without holding | |
| 8897 // the lock, we are assured that it will not change | |
| 8898 // because we "own" this oop, so no other thread can | |
| 8899 // be trying to push it on the overflow list; see | |
| 8900 // the assertion in preserve_mark_work() that checks | |
| 8901 // that m == p->mark(). | |
| 8902 preserve_mark_work(p, m); | |
| 8903 } | |
| 8904 } | |
| 8905 | |
| 8906 // We should be able to do this multi-threaded, | |
| 8907 // a chunk of stack being a task (this is | |
| 8908 // correct because each oop only ever appears | |
| 8909 // once in the overflow list. However, it's | |
| 8910 // not very easy to completely overlap this with | |
| 8911 // other operations, so will generally not be done | |
| 8912 // until all work's been completed. Because we | |
| 8913 // expect the preserved oop stack (set) to be small, | |
| 8914 // it's probably fine to do this single-threaded. | |
| 8915 // We can explore cleverer concurrent/overlapped/parallel | |
| 8916 // processing of preserved marks if we feel the | |
| 8917 // need for this in the future. Stack overflow should | |
| 8918 // be so rare in practice and, when it happens, its | |
| 8919 // effect on performance so great that this will | |
| 8920 // likely just be in the noise anyway. | |
| 8921 void CMSCollector::restore_preserved_marks_if_any() { | |
| 8922 if (_preserved_oop_stack == NULL) { | |
| 8923 assert(_preserved_mark_stack == NULL, | |
| 8924 "bijection with preserved_oop_stack"); | |
| 8925 return; | |
| 8926 } | |
| 8927 | |
| 8928 assert(SafepointSynchronize::is_at_safepoint(), | |
| 8929 "world should be stopped"); | |
| 8930 assert(Thread::current()->is_ConcurrentGC_thread() || | |
| 8931 Thread::current()->is_VM_thread(), | |
| 8932 "should be single-threaded"); | |
| 8933 | |
| 8934 int length = _preserved_oop_stack->length(); | |
| 8935 assert(_preserved_mark_stack->length() == length, "bijection"); | |
| 8936 for (int i = 0; i < length; i++) { | |
| 8937 oop p = _preserved_oop_stack->at(i); | |
| 8938 assert(p->is_oop(), "Should be an oop"); | |
| 8939 assert(_span.contains(p), "oop should be in _span"); | |
| 8940 assert(p->mark() == markOopDesc::prototype(), | |
| 8941 "Set when taken from overflow list"); | |
| 8942 markOop m = _preserved_mark_stack->at(i); | |
| 8943 p->set_mark(m); | |
| 8944 } | |
| 8945 _preserved_mark_stack->clear(); | |
| 8946 _preserved_oop_stack->clear(); | |
| 8947 assert(_preserved_mark_stack->is_empty() && | |
| 8948 _preserved_oop_stack->is_empty(), | |
| 8949 "stacks were cleared above"); | |
| 8950 } | |
| 8951 | |
| 8952 #ifndef PRODUCT | |
| 8953 bool CMSCollector::no_preserved_marks() const { | |
| 8954 return ( ( _preserved_mark_stack == NULL | |
| 8955 && _preserved_oop_stack == NULL) | |
| 8956 || ( _preserved_mark_stack->is_empty() | |
| 8957 && _preserved_oop_stack->is_empty())); | |
| 8958 } | |
| 8959 #endif | |
| 8960 | |
| 8961 CMSAdaptiveSizePolicy* ASConcurrentMarkSweepGeneration::cms_size_policy() const | |
| 8962 { | |
| 8963 GenCollectedHeap* gch = (GenCollectedHeap*) GenCollectedHeap::heap(); | |
| 8964 CMSAdaptiveSizePolicy* size_policy = | |
| 8965 (CMSAdaptiveSizePolicy*) gch->gen_policy()->size_policy(); | |
| 8966 assert(size_policy->is_gc_cms_adaptive_size_policy(), | |
| 8967 "Wrong type for size policy"); | |
| 8968 return size_policy; | |
| 8969 } | |
| 8970 | |
| 8971 void ASConcurrentMarkSweepGeneration::resize(size_t cur_promo_size, | |
| 8972 size_t desired_promo_size) { | |
| 8973 if (cur_promo_size < desired_promo_size) { | |
| 8974 size_t expand_bytes = desired_promo_size - cur_promo_size; | |
| 8975 if (PrintAdaptiveSizePolicy && Verbose) { | |
| 8976 gclog_or_tty->print_cr(" ASConcurrentMarkSweepGeneration::resize " | |
| 8977 "Expanding tenured generation by " SIZE_FORMAT " (bytes)", | |
| 8978 expand_bytes); | |
| 8979 } | |
| 8980 expand(expand_bytes, | |
| 8981 MinHeapDeltaBytes, | |
| 8982 CMSExpansionCause::_adaptive_size_policy); | |
| 8983 } else if (desired_promo_size < cur_promo_size) { | |
| 8984 size_t shrink_bytes = cur_promo_size - desired_promo_size; | |
| 8985 if (PrintAdaptiveSizePolicy && Verbose) { | |
| 8986 gclog_or_tty->print_cr(" ASConcurrentMarkSweepGeneration::resize " | |
| 8987 "Shrinking tenured generation by " SIZE_FORMAT " (bytes)", | |
| 8988 shrink_bytes); | |
| 8989 } | |
| 8990 shrink(shrink_bytes); | |
| 8991 } | |
| 8992 } | |
| 8993 | |
| 8994 CMSGCAdaptivePolicyCounters* ASConcurrentMarkSweepGeneration::gc_adaptive_policy_counters() { | |
| 8995 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 8996 CMSGCAdaptivePolicyCounters* counters = | |
| 8997 (CMSGCAdaptivePolicyCounters*) gch->collector_policy()->counters(); | |
| 8998 assert(counters->kind() == GCPolicyCounters::CMSGCAdaptivePolicyCountersKind, | |
| 8999 "Wrong kind of counters"); | |
| 9000 return counters; | |
| 9001 } | |
| 9002 | |
| 9003 | |
| 9004 void ASConcurrentMarkSweepGeneration::update_counters() { | |
| 9005 if (UsePerfData) { | |
| 9006 _space_counters->update_all(); | |
| 9007 _gen_counters->update_all(); | |
| 9008 CMSGCAdaptivePolicyCounters* counters = gc_adaptive_policy_counters(); | |
| 9009 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 9010 CMSGCStats* gc_stats_l = (CMSGCStats*) gc_stats(); | |
| 9011 assert(gc_stats_l->kind() == GCStats::CMSGCStatsKind, | |
| 9012 "Wrong gc statistics type"); | |
| 9013 counters->update_counters(gc_stats_l); | |
| 9014 } | |
| 9015 } | |
| 9016 | |
| 9017 void ASConcurrentMarkSweepGeneration::update_counters(size_t used) { | |
| 9018 if (UsePerfData) { | |
| 9019 _space_counters->update_used(used); | |
| 9020 _space_counters->update_capacity(); | |
| 9021 _gen_counters->update_all(); | |
| 9022 | |
| 9023 CMSGCAdaptivePolicyCounters* counters = gc_adaptive_policy_counters(); | |
| 9024 GenCollectedHeap* gch = GenCollectedHeap::heap(); | |
| 9025 CMSGCStats* gc_stats_l = (CMSGCStats*) gc_stats(); | |
| 9026 assert(gc_stats_l->kind() == GCStats::CMSGCStatsKind, | |
| 9027 "Wrong gc statistics type"); | |
| 9028 counters->update_counters(gc_stats_l); | |
| 9029 } | |
| 9030 } | |
| 9031 | |
| 9032 // The desired expansion delta is computed so that: | |
| 9033 // . desired free percentage or greater is used | |
| 9034 void ASConcurrentMarkSweepGeneration::compute_new_size() { | |
| 9035 assert_locked_or_safepoint(Heap_lock); | |
| 9036 | |
| 9037 GenCollectedHeap* gch = (GenCollectedHeap*) GenCollectedHeap::heap(); | |
| 9038 | |
| 9039 // If incremental collection failed, we just want to expand | |
| 9040 // to the limit. | |
| 9041 if (incremental_collection_failed()) { | |
| 9042 clear_incremental_collection_failed(); | |
| 9043 grow_to_reserved(); | |
| 9044 return; | |
| 9045 } | |
| 9046 | |
| 9047 assert(UseAdaptiveSizePolicy, "Should be using adaptive sizing"); | |
| 9048 | |
| 9049 assert(gch->kind() == CollectedHeap::GenCollectedHeap, | |
| 9050 "Wrong type of heap"); | |
| 9051 int prev_level = level() - 1; | |
| 9052 assert(prev_level >= 0, "The cms generation is the lowest generation"); | |
| 9053 Generation* prev_gen = gch->get_gen(prev_level); | |
| 9054 assert(prev_gen->kind() == Generation::ASParNew, | |
| 9055 "Wrong type of young generation"); | |
| 9056 ParNewGeneration* younger_gen = (ParNewGeneration*) prev_gen; | |
| 9057 size_t cur_eden = younger_gen->eden()->capacity(); | |
| 9058 CMSAdaptiveSizePolicy* size_policy = cms_size_policy(); | |
| 9059 size_t cur_promo = free(); | |
| 9060 size_policy->compute_tenured_generation_free_space(cur_promo, | |
| 9061 max_available(), | |
| 9062 cur_eden); | |
| 9063 resize(cur_promo, size_policy->promo_size()); | |
| 9064 | |
| 9065 // Record the new size of the space in the cms generation | |
| 9066 // that is available for promotions. This is temporary. | |
| 9067 // It should be the desired promo size. | |
| 9068 size_policy->avg_cms_promo()->sample(free()); | |
| 9069 size_policy->avg_old_live()->sample(used()); | |
| 9070 | |
| 9071 if (UsePerfData) { | |
| 9072 CMSGCAdaptivePolicyCounters* counters = gc_adaptive_policy_counters(); | |
| 9073 counters->update_cms_capacity_counter(capacity()); | |
| 9074 } | |
| 9075 } | |
| 9076 | |
| 9077 void ASConcurrentMarkSweepGeneration::shrink_by(size_t desired_bytes) { | |
| 9078 assert_locked_or_safepoint(Heap_lock); | |
| 9079 assert_lock_strong(freelistLock()); | |
| 9080 HeapWord* old_end = _cmsSpace->end(); | |
| 9081 HeapWord* unallocated_start = _cmsSpace->unallocated_block(); | |
| 9082 assert(old_end >= unallocated_start, "Miscalculation of unallocated_start"); | |
| 9083 FreeChunk* chunk_at_end = find_chunk_at_end(); | |
| 9084 if (chunk_at_end == NULL) { | |
| 9085 // No room to shrink | |
| 9086 if (PrintGCDetails && Verbose) { | |
| 9087 gclog_or_tty->print_cr("No room to shrink: old_end " | |
| 9088 PTR_FORMAT " unallocated_start " PTR_FORMAT | |
| 9089 " chunk_at_end " PTR_FORMAT, | |
| 9090 old_end, unallocated_start, chunk_at_end); | |
| 9091 } | |
| 9092 return; | |
| 9093 } else { | |
| 9094 | |
| 9095 // Find the chunk at the end of the space and determine | |
| 9096 // how much it can be shrunk. | |
| 9097 size_t shrinkable_size_in_bytes = chunk_at_end->size(); | |
| 9098 size_t aligned_shrinkable_size_in_bytes = | |
| 9099 align_size_down(shrinkable_size_in_bytes, os::vm_page_size()); | |
| 9100 assert(unallocated_start <= chunk_at_end->end(), | |
| 9101 "Inconsistent chunk at end of space"); | |
| 9102 size_t bytes = MIN2(desired_bytes, aligned_shrinkable_size_in_bytes); | |
| 9103 size_t word_size_before = heap_word_size(_virtual_space.committed_size()); | |
| 9104 | |
| 9105 // Shrink the underlying space | |
| 9106 _virtual_space.shrink_by(bytes); | |
| 9107 if (PrintGCDetails && Verbose) { | |
| 9108 gclog_or_tty->print_cr("ConcurrentMarkSweepGeneration::shrink_by:" | |
| 9109 " desired_bytes " SIZE_FORMAT | |
| 9110 " shrinkable_size_in_bytes " SIZE_FORMAT | |
| 9111 " aligned_shrinkable_size_in_bytes " SIZE_FORMAT | |
| 9112 " bytes " SIZE_FORMAT, | |
| 9113 desired_bytes, shrinkable_size_in_bytes, | |
| 9114 aligned_shrinkable_size_in_bytes, bytes); | |
| 9115 gclog_or_tty->print_cr(" old_end " SIZE_FORMAT | |
| 9116 " unallocated_start " SIZE_FORMAT, | |
| 9117 old_end, unallocated_start); | |
| 9118 } | |
| 9119 | |
| 9120 // If the space did shrink (shrinking is not guaranteed), | |
| 9121 // shrink the chunk at the end by the appropriate amount. | |
| 9122 if (((HeapWord*)_virtual_space.high()) < old_end) { | |
| 9123 size_t new_word_size = | |
| 9124 heap_word_size(_virtual_space.committed_size()); | |
| 9125 | |
| 9126 // Have to remove the chunk from the dictionary because it is changing | |
| 9127 // size and might be someplace elsewhere in the dictionary. | |
| 9128 | |
| 9129 // Get the chunk at end, shrink it, and put it | |
| 9130 // back. | |
| 9131 _cmsSpace->removeChunkFromDictionary(chunk_at_end); | |
| 9132 size_t word_size_change = word_size_before - new_word_size; | |
| 9133 size_t chunk_at_end_old_size = chunk_at_end->size(); | |
| 9134 assert(chunk_at_end_old_size >= word_size_change, | |
| 9135 "Shrink is too large"); | |
| 9136 chunk_at_end->setSize(chunk_at_end_old_size - | |
| 9137 word_size_change); | |
| 9138 _cmsSpace->freed((HeapWord*) chunk_at_end->end(), | |
| 9139 word_size_change); | |
| 9140 | |
| 9141 _cmsSpace->returnChunkToDictionary(chunk_at_end); | |
| 9142 | |
| 9143 MemRegion mr(_cmsSpace->bottom(), new_word_size); | |
| 9144 _bts->resize(new_word_size); // resize the block offset shared array | |
| 9145 Universe::heap()->barrier_set()->resize_covered_region(mr); | |
| 9146 _cmsSpace->assert_locked(); | |
| 9147 _cmsSpace->set_end((HeapWord*)_virtual_space.high()); | |
| 9148 | |
| 9149 NOT_PRODUCT(_cmsSpace->dictionary()->verify()); | |
| 9150 | |
| 9151 // update the space and generation capacity counters | |
| 9152 if (UsePerfData) { | |
| 9153 _space_counters->update_capacity(); | |
| 9154 _gen_counters->update_all(); | |
| 9155 } | |
| 9156 | |
| 9157 if (Verbose && PrintGCDetails) { | |
| 9158 size_t new_mem_size = _virtual_space.committed_size(); | |
| 9159 size_t old_mem_size = new_mem_size + bytes; | |
| 9160 gclog_or_tty->print_cr("Shrinking %s from %ldK by %ldK to %ldK", | |
| 9161 name(), old_mem_size/K, bytes/K, new_mem_size/K); | |
| 9162 } | |
| 9163 } | |
| 9164 | |
| 9165 assert(_cmsSpace->unallocated_block() <= _cmsSpace->end(), | |
| 9166 "Inconsistency at end of space"); | |
| 9167 assert(chunk_at_end->end() == _cmsSpace->end(), | |
| 9168 "Shrinking is inconsistent"); | |
| 9169 return; | |
| 9170 } | |
| 9171 } | |
| 9172 | |
| 9173 // Transfer some number of overflown objects to usual marking | |
| 9174 // stack. Return true if some objects were transferred. | |
| 9175 bool MarkRefsIntoAndScanClosure::take_from_overflow_list() { | |
|
679
cea947c8a988
6819891: ParNew: Fix work queue overflow code to deal correctly with +UseCompressedOops
ysr
parents:
628
diff
changeset
|
9176 size_t num = MIN2((size_t)(_mark_stack->capacity() - _mark_stack->length())/4, |
| 0 | 9177 (size_t)ParGCDesiredObjsFromOverflowList); |
| 9178 | |
| 9179 bool res = _collector->take_from_overflow_list(num, _mark_stack); | |
| 9180 assert(_collector->overflow_list_is_empty() || res, | |
| 9181 "If list is not empty, we should have taken something"); | |
| 9182 assert(!res || !_mark_stack->isEmpty(), | |
| 9183 "If we took something, it should now be on our stack"); | |
| 9184 return res; | |
| 9185 } | |
| 9186 | |
| 9187 size_t MarkDeadObjectsClosure::do_blk(HeapWord* addr) { | |
| 9188 size_t res = _sp->block_size_no_stall(addr, _collector); | |
| 9189 assert(res != 0, "Should always be able to compute a size"); | |
| 9190 if (_sp->block_is_obj(addr)) { | |
| 9191 if (_live_bit_map->isMarked(addr)) { | |
| 9192 // It can't have been dead in a previous cycle | |
| 9193 guarantee(!_dead_bit_map->isMarked(addr), "No resurrection!"); | |
| 9194 } else { | |
| 9195 _dead_bit_map->mark(addr); // mark the dead object | |
| 9196 } | |
| 9197 } | |
| 9198 return res; | |
| 9199 } | |
|
1703
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9200 |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9201 TraceCMSMemoryManagerStats::TraceCMSMemoryManagerStats(CMSCollector::CollectorState phase): TraceMemoryManagerStats() { |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9202 |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9203 switch (phase) { |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9204 case CMSCollector::InitialMarking: |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9205 initialize(true /* fullGC */ , |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9206 true /* recordGCBeginTime */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9207 true /* recordPreGCUsage */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9208 false /* recordPeakUsage */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9209 false /* recordPostGCusage */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9210 true /* recordAccumulatedGCTime */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9211 false /* recordGCEndTime */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9212 false /* countCollection */ ); |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9213 break; |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9214 |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9215 case CMSCollector::FinalMarking: |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9216 initialize(true /* fullGC */ , |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9217 false /* recordGCBeginTime */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9218 false /* recordPreGCUsage */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9219 false /* recordPeakUsage */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9220 false /* recordPostGCusage */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9221 true /* recordAccumulatedGCTime */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9222 false /* recordGCEndTime */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9223 false /* countCollection */ ); |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9224 break; |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9225 |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9226 case CMSCollector::Sweeping: |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9227 initialize(true /* fullGC */ , |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9228 false /* recordGCBeginTime */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9229 false /* recordPreGCUsage */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9230 true /* recordPeakUsage */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9231 true /* recordPostGCusage */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9232 false /* recordAccumulatedGCTime */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9233 true /* recordGCEndTime */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9234 true /* countCollection */ ); |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9235 break; |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9236 |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9237 default: |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9238 ShouldNotReachHere(); |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9239 } |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9240 } |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9241 |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9242 // when bailing out of cms in concurrent mode failure |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9243 TraceCMSMemoryManagerStats::TraceCMSMemoryManagerStats(): TraceMemoryManagerStats() { |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9244 initialize(true /* fullGC */ , |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9245 true /* recordGCBeginTime */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9246 true /* recordPreGCUsage */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9247 true /* recordPeakUsage */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9248 true /* recordPostGCusage */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9249 true /* recordAccumulatedGCTime */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9250 true /* recordGCEndTime */, |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9251 true /* countCollection */ ); |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9252 } |
|
f6f3eef8a521
6581734: CMS Old Gen's collection usage is zero after GC which is incorrect
kevinw
parents:
1579
diff
changeset
|
9253 |