graal-compiler: src/share/vm/gc_implementation/g1/concurrentG1Refine.cpp annotate

annotate src/share/vm/gc_implementation/g1/concurrentG1Refine.cpp @ 1974:fd1d227ef1b9

6983204: G1: Nightly test nsk/regression/b4958615 failing with +ExplicitGCInvokesConcurrent Summary: Enable reference discovery during concurrent marking by setting the reference processor field of the concurrent marking closure. Keep reference objects on the discovered reference lists alive during incremental evacuation pauses until they are processed at the end of concurrent marking. Reviewed-by: ysr, tonyp

author	johnc
date	Wed, 01 Dec 2010 17:34:02 -0800
parents	f95d63e2154a
children	c33825b68624

rev	line source
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	1 /*
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	2 * Copyright (c) 2001, 2010, Oracle and/or its affiliates. All rights reserved.
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	4 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	5 * This code is free software; you can redistribute it and/or modify it
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	6 * under the terms of the GNU General Public License version 2 only, as
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	7 * published by the Free Software Foundation.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	8 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	9 * This code is distributed in the hope that it will be useful, but WITHOUT
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	10 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	11 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	12 * version 2 for more details (a copy is included in the LICENSE file that
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	13 * accompanied this code).
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	14 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	15 * You should have received a copy of the GNU General Public License version
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	16 * 2 along with this work; if not, write to the Free Software Foundation,
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	17 * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	18 *
1552 c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1282 diff changeset	19 * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1282 diff changeset	20 * or visit www.oracle.com if you need additional information or have any
c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1282 diff changeset	21 * questions.
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	22 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	23 */
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	24
1972 f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	25 #include "precompiled.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	26 #include "gc_implementation/g1/concurrentG1Refine.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	27 #include "gc_implementation/g1/concurrentG1RefineThread.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	28 #include "gc_implementation/g1/g1CollectedHeap.inline.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	29 #include "gc_implementation/g1/g1CollectorPolicy.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	30 #include "gc_implementation/g1/g1RemSet.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	31 #include "gc_implementation/g1/heapRegionSeq.inline.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	32 #include "memory/space.inline.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	33 #include "runtime/atomic.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	34 #include "utilities/copy.hpp"
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	35
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	36 // Possible sizes for the card counts cache: odd primes that roughly double in size.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	37 // (See jvmtiTagMap.cpp).
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	38 int ConcurrentG1Refine::_cc_cache_sizes[] = {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	39 16381, 32771, 76831, 150001, 307261,
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	40 614563, 1228891, 2457733, 4915219, 9830479,
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	41 19660831, 39321619, 78643219, 157286461, -1
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	42 };
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	43
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	44 ConcurrentG1Refine::ConcurrentG1Refine() :
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	45 _card_counts(NULL), _card_epochs(NULL),
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	46 _n_card_counts(0), _max_n_card_counts(0),
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	47 _cache_size_index(0), _expand_card_counts(false),
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	48 _hot_cache(NULL),
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	49 _def_use_cache(false), _use_cache(false),
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	50 _n_periods(0),
794 315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	51 _threads(NULL), _n_threads(0)
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	52 {
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	53
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	54 // Ergomonically select initial concurrent refinement parameters
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	55 if (FLAG_IS_DEFAULT(G1ConcRefinementGreenZone)) {
b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	56 FLAG_SET_DEFAULT(G1ConcRefinementGreenZone, MAX2<int>(ParallelGCThreads, 1));
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	57 }
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	58 set_green_zone(G1ConcRefinementGreenZone);
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	59
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	60 if (FLAG_IS_DEFAULT(G1ConcRefinementYellowZone)) {
b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	61 FLAG_SET_DEFAULT(G1ConcRefinementYellowZone, green_zone() * 3);
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	62 }
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	63 set_yellow_zone(MAX2<int>(G1ConcRefinementYellowZone, green_zone()));
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	64
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	65 if (FLAG_IS_DEFAULT(G1ConcRefinementRedZone)) {
b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	66 FLAG_SET_DEFAULT(G1ConcRefinementRedZone, yellow_zone() * 2);
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	67 }
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	68 set_red_zone(MAX2<int>(G1ConcRefinementRedZone, yellow_zone()));
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	69 _n_worker_threads = thread_num();
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	70 // We need one extra thread to do the young gen rset size sampling.
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	71 _n_threads = _n_worker_threads + 1;
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	72 reset_threshold_step();
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	73
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	74 _threads = NEW_C_HEAP_ARRAY(ConcurrentG1RefineThread*, _n_threads);
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	75 int worker_id_offset = (int)DirtyCardQueueSet::num_par_ids();
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	76 ConcurrentG1RefineThread *next = NULL;
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	77 for (int i = _n_threads - 1; i >= 0; i--) {
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	78 ConcurrentG1RefineThread* t = new ConcurrentG1RefineThread(this, next, worker_id_offset, i);
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	79 assert(t != NULL, "Conc refine should have been created");
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	80 assert(t->cg1r() == this, "Conc refine thread should refer to this");
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	81 _threads[i] = t;
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	82 next = t;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	83 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	84 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	85
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	86 void ConcurrentG1Refine::reset_threshold_step() {
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	87 if (FLAG_IS_DEFAULT(G1ConcRefinementThresholdStep)) {
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	88 _thread_threshold_step = (yellow_zone() - green_zone()) / (worker_thread_num() + 1);
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	89 } else {
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	90 _thread_threshold_step = G1ConcRefinementThresholdStep;
795 215f81b4d9b3 6841831: G1: assert(contains_reference(from),"We just added it!") fires iveresov parents: 794 diff changeset	91 }
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	92 }
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	93
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	94 int ConcurrentG1Refine::thread_num() {
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	95 return MAX2<int>((G1ConcRefinementThreads > 0) ? G1ConcRefinementThreads : ParallelGCThreads, 1);
795 215f81b4d9b3 6841831: G1: assert(contains_reference(from),"We just added it!") fires iveresov parents: 794 diff changeset	96 }
215f81b4d9b3 6841831: G1: assert(contains_reference(from),"We just added it!") fires iveresov parents: 794 diff changeset	97
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	98 void ConcurrentG1Refine::init() {
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	99 if (G1ConcRSLogCacheSize > 0) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	100 _g1h = G1CollectedHeap::heap();
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	101 _max_n_card_counts =
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	102 (unsigned) (_g1h->g1_reserved_obj_bytes() >> CardTableModRefBS::card_shift);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	103
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	104 size_t max_card_num = ((size_t)1 << (sizeof(unsigned)*BitsPerByte-1)) - 1;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	105 guarantee(_max_n_card_counts < max_card_num, "card_num representation");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	106
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	107 int desired = _max_n_card_counts / InitialCacheFraction;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	108 for (_cache_size_index = 0;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	109 _cc_cache_sizes[_cache_size_index] >= 0; _cache_size_index++) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	110 if (_cc_cache_sizes[_cache_size_index] >= desired) break;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	111 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	112 _cache_size_index = MAX2(0, (_cache_size_index - 1));
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	113
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	114 int initial_size = _cc_cache_sizes[_cache_size_index];
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	115 if (initial_size < 0) initial_size = _max_n_card_counts;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	116
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	117 // Make sure we don't go bigger than we will ever need
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	118 _n_card_counts = MIN2((unsigned) initial_size, _max_n_card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	119
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	120 _card_counts = NEW_C_HEAP_ARRAY(CardCountCacheEntry, _n_card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	121 _card_epochs = NEW_C_HEAP_ARRAY(CardEpochCacheEntry, _n_card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	122
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	123 Copy::fill_to_bytes(&_card_counts[0],
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	124 _n_card_counts * sizeof(CardCountCacheEntry));
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	125 Copy::fill_to_bytes(&_card_epochs[0], _n_card_counts * sizeof(CardEpochCacheEntry));
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	126
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	127 ModRefBarrierSet* bs = _g1h->mr_bs();
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	128 guarantee(bs->is_a(BarrierSet::CardTableModRef), "Precondition");
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	129 _ct_bs = (CardTableModRefBS*)bs;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	130 _ct_bot = _ct_bs->byte_for_const(_g1h->reserved_region().start());
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	131
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	132 _def_use_cache = true;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	133 _use_cache = true;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	134 _hot_cache_size = (1 << G1ConcRSLogCacheSize);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	135 _hot_cache = NEW_C_HEAP_ARRAY(jbyte*, _hot_cache_size);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	136 _n_hot = 0;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	137 _hot_cache_idx = 0;
889 15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	138
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	139 // For refining the cards in the hot cache in parallel
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	140 int n_workers = (ParallelGCThreads > 0 ?
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	141 _g1h->workers()->total_workers() : 1);
889 15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	142 _hot_cache_par_chunk_size = MAX2(1, _hot_cache_size / n_workers);
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	143 _hot_cache_par_claimed_idx = 0;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	144 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	145 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	146
794 315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	147 void ConcurrentG1Refine::stop() {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	148 if (_threads != NULL) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	149 for (int i = 0; i < _n_threads; i++) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	150 _threads[i]->stop();
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	151 }
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	152 }
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	153 }
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	154
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	155 void ConcurrentG1Refine::reinitialize_threads() {
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	156 reset_threshold_step();
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	157 if (_threads != NULL) {
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	158 for (int i = 0; i < _n_threads; i++) {
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	159 _threads[i]->initialize();
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	160 }
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	161 }
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	162 }
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	163
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	164 ConcurrentG1Refine::~ConcurrentG1Refine() {
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	165 if (G1ConcRSLogCacheSize > 0) {
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	166 assert(_card_counts != NULL, "Logic");
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	167 FREE_C_HEAP_ARRAY(CardCountCacheEntry, _card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	168 assert(_card_epochs != NULL, "Logic");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	169 FREE_C_HEAP_ARRAY(CardEpochCacheEntry, _card_epochs);
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	170 assert(_hot_cache != NULL, "Logic");
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	171 FREE_C_HEAP_ARRAY(jbyte*, _hot_cache);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	172 }
794 315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	173 if (_threads != NULL) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	174 for (int i = 0; i < _n_threads; i++) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	175 delete _threads[i];
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	176 }
799 f89cf529c3c7 6849122: G1: Typo introduced during implementation of the parallel refinement iveresov parents: 795 diff changeset	177 FREE_C_HEAP_ARRAY(ConcurrentG1RefineThread*, _threads);
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	178 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	179 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	180
794 315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	181 void ConcurrentG1Refine::threads_do(ThreadClosure *tc) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	182 if (_threads != NULL) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	183 for (int i = 0; i < _n_threads; i++) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	184 tc->do_thread(_threads[i]);
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	185 }
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	186 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	187 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	188
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	189 bool ConcurrentG1Refine::is_young_card(jbyte* card_ptr) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	190 HeapWord* start = _ct_bs->addr_for(card_ptr);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	191 HeapRegion* r = _g1h->heap_region_containing(start);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	192 if (r != NULL && r->is_young()) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	193 return true;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	194 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	195 // This card is not associated with a heap region
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	196 // so can't be young.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	197 return false;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	198 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	199
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	200 jbyte* ConcurrentG1Refine::add_card_count(jbyte* card_ptr, int* count, bool* defer) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	201 unsigned new_card_num = ptr_2_card_num(card_ptr);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	202 unsigned bucket = hash(new_card_num);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	203 assert(0 <= bucket && bucket < _n_card_counts, "Bounds");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	204
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	205 CardCountCacheEntry* count_ptr = &_card_counts[bucket];
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	206 CardEpochCacheEntry* epoch_ptr = &_card_epochs[bucket];
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	207
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	208 // We have to construct a new entry if we haven't updated the counts
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	209 // during the current period, or if the count was updated for a
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	210 // different card number.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	211 unsigned int new_epoch = (unsigned int) _n_periods;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	212 julong new_epoch_entry = make_epoch_entry(new_card_num, new_epoch);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	213
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	214 while (true) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	215 // Fetch the previous epoch value
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	216 julong prev_epoch_entry = epoch_ptr->_value;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	217 julong cas_res;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	218
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	219 if (extract_epoch(prev_epoch_entry) != new_epoch) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	220 // This entry has not yet been updated during this period.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	221 // Note: we update the epoch value atomically to ensure
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	222 // that there is only one winner that updates the cached
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	223 // card_ptr value even though all the refine threads share
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	224 // the same epoch value.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	225
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	226 cas_res = (julong) Atomic::cmpxchg((jlong) new_epoch_entry,
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	227 (volatile jlong*)&epoch_ptr->_value,
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	228 (jlong) prev_epoch_entry);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	229
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	230 if (cas_res == prev_epoch_entry) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	231 // We have successfully won the race to update the
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	232 // epoch and card_num value. Make it look like the
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	233 // count and eviction count were previously cleared.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	234 count_ptr->_count = 1;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	235 count_ptr->_evict_count = 0;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	236 *count = 0;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	237 // We can defer the processing of card_ptr
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	238 *defer = true;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	239 return card_ptr;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	240 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	241 // We did not win the race to update the epoch field, so some other
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	242 // thread must have done it. The value that gets returned by CAS
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	243 // should be the new epoch value.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	244 assert(extract_epoch(cas_res) == new_epoch, "unexpected epoch");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	245 // We could 'continue' here or just re-read the previous epoch value
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	246 prev_epoch_entry = epoch_ptr->_value;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	247 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	248
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	249 // The epoch entry for card_ptr has been updated during this period.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	250 unsigned old_card_num = extract_card_num(prev_epoch_entry);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	251
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	252 // The card count that will be returned to caller
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	253 *count = count_ptr->_count;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	254
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	255 // Are we updating the count for the same card?
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	256 if (new_card_num == old_card_num) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	257 // Same card - just update the count. We could have more than one
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	258 // thread racing to update count for the current card. It should be
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	259 // OK not to use a CAS as the only penalty should be some missed
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	260 // increments of the count which delays identifying the card as "hot".
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	261
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	262 if (*count < max_jubyte) count_ptr->_count++;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	263 // We can defer the processing of card_ptr
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	264 *defer = true;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	265 return card_ptr;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	266 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	267
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	268 // Different card - evict old card info
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	269 if (count_ptr->_evict_count < max_jubyte) count_ptr->_evict_count++;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	270 if (count_ptr->_evict_count > G1CardCountCacheExpandThreshold) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	271 // Trigger a resize the next time we clear
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	272 _expand_card_counts = true;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	273 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	274
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	275 cas_res = (julong) Atomic::cmpxchg((jlong) new_epoch_entry,
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	276 (volatile jlong*)&epoch_ptr->_value,
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	277 (jlong) prev_epoch_entry);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	278
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	279 if (cas_res == prev_epoch_entry) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	280 // We successfully updated the card num value in the epoch entry
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	281 count_ptr->_count = 0; // initialize counter for new card num
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	282 jbyte* old_card_ptr = card_num_2_ptr(old_card_num);
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	283
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	284 // Even though the region containg the card at old_card_num was not
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	285 // in the young list when old_card_num was recorded in the epoch
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	286 // cache it could have been added to the free list and subsequently
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	287 // added to the young list in the intervening time. See CR 6817995.
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	288 // We do not deal with this case here - it will be handled in
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	289 // HeapRegion::oops_on_card_seq_iterate_careful after it has been
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	290 // determined that the region containing the card has been allocated
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	291 // to, and it's safe to check the young type of the region.
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	292
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	293 // We do not want to defer processing of card_ptr in this case
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	294 // (we need to refine old_card_ptr and card_ptr)
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	295 *defer = false;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	296 return old_card_ptr;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	297 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	298 // Someone else beat us - try again.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	299 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	300 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	301
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	302 jbyte* ConcurrentG1Refine::cache_insert(jbyte* card_ptr, bool* defer) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	303 int count;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	304 jbyte* cached_ptr = add_card_count(card_ptr, &count, defer);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	305 assert(cached_ptr != NULL, "bad cached card ptr");
1246 deada8912c54 6914402: G1: assert(!is_young_card(cached_ptr),"shouldn't get a card in young region") johnc parents: 1111 diff changeset	306
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	307 // We've just inserted a card pointer into the card count cache
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	308 // and got back the card that we just inserted or (evicted) the
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	309 // previous contents of that count slot.
1246 deada8912c54 6914402: G1: assert(!is_young_card(cached_ptr),"shouldn't get a card in young region") johnc parents: 1111 diff changeset	310
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	311 // The card we got back could be in a young region. When the
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	312 // returned card (if evicted) was originally inserted, we had
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	313 // determined that its containing region was not young. However
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	314 // it is possible for the region to be freed during a cleanup
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	315 // pause, then reallocated and tagged as young which will result
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	316 // in the returned card residing in a young region.
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	317 //
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	318 // We do not deal with this case here - the change from non-young
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	319 // to young could be observed at any time - it will be handled in
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	320 // HeapRegion::oops_on_card_seq_iterate_careful after it has been
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	321 // determined that the region containing the card has been allocated
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	322 // to.
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	323
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	324 // The card pointer we obtained from card count cache is not hot
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	325 // so do not store it in the cache; return it for immediate
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	326 // refining.
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	327 if (count < G1ConcRSHotCardLimit) {
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	328 return cached_ptr;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	329 }
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	330
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	331 // Otherwise, the pointer we got from the _card_counts cache is hot.
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	332 jbyte* res = NULL;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	333 MutexLockerEx x(HotCardCache_lock, Mutex::_no_safepoint_check_flag);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	334 if (_n_hot == _hot_cache_size) {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	335 res = _hot_cache[_hot_cache_idx];
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	336 _n_hot--;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	337 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	338 // Now _n_hot < _hot_cache_size, and we can insert at _hot_cache_idx.
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	339 _hot_cache[_hot_cache_idx] = cached_ptr;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	340 _hot_cache_idx++;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	341 if (_hot_cache_idx == _hot_cache_size) _hot_cache_idx = 0;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	342 _n_hot++;
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	343
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	344 // The card obtained from the hot card cache could be in a young
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	345 // region. See above on how this can happen.
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	346
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	347 return res;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	348 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	349
1705 2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	350 void ConcurrentG1Refine::clean_up_cache(int worker_i,
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	351 G1RemSet* g1rs,
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	352 DirtyCardQueue* into_cset_dcq) {
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	353 assert(!use_cache(), "cache should be disabled");
889 15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	354 int start_idx;
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	355
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	356 while ((start_idx = _hot_cache_par_claimed_idx) < _n_hot) { // read once
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	357 int end_idx = start_idx + _hot_cache_par_chunk_size;
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	358
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	359 if (start_idx ==
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	360 Atomic::cmpxchg(end_idx, &_hot_cache_par_claimed_idx, start_idx)) {
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	361 // The current worker has successfully claimed the chunk [start_idx..end_idx)
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	362 end_idx = MIN2(end_idx, _n_hot);
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	363 for (int i = start_idx; i < end_idx; i++) {
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	364 jbyte* entry = _hot_cache[i];
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	365 if (entry != NULL) {
1705 2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	366 if (g1rs->concurrentRefineOneCard(entry, worker_i, true)) {
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	367 // 'entry' contains references that point into the current
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	368 // collection set. We need to record 'entry' in the DCQS
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	369 // that's used for that purpose.
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	370 //
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	371 // The only time we care about recording cards that contain
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	372 // references that point into the collection set is during
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	373 // RSet updating while within an evacuation pause.
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	374 // In this case worker_i should be the id of a GC worker thread
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	375 assert(SafepointSynchronize::is_at_safepoint(), "not during an evacuation pause");
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	376 assert(worker_i < (int) DirtyCardQueueSet::num_par_ids(), "incorrect worker id");
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	377 into_cset_dcq->enqueue(entry);
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	378 }
889 15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	379 }
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	380 }
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	381 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	382 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	383 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	384
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	385 void ConcurrentG1Refine::expand_card_count_cache() {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	386 if (_n_card_counts < _max_n_card_counts) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	387 int new_idx = _cache_size_index+1;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	388 int new_size = _cc_cache_sizes[new_idx];
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	389 if (new_size < 0) new_size = _max_n_card_counts;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	390
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	391 // Make sure we don't go bigger than we will ever need
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	392 new_size = MIN2((unsigned) new_size, _max_n_card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	393
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	394 // Expand the card count and card epoch tables
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	395 if (new_size > (int)_n_card_counts) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	396 // We can just free and allocate a new array as we're
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	397 // not interested in preserving the contents
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	398 assert(_card_counts != NULL, "Logic!");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	399 assert(_card_epochs != NULL, "Logic!");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	400 FREE_C_HEAP_ARRAY(CardCountCacheEntry, _card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	401 FREE_C_HEAP_ARRAY(CardEpochCacheEntry, _card_epochs);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	402 _n_card_counts = new_size;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	403 _card_counts = NEW_C_HEAP_ARRAY(CardCountCacheEntry, _n_card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	404 _card_epochs = NEW_C_HEAP_ARRAY(CardEpochCacheEntry, _n_card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	405 _cache_size_index = new_idx;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	406 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	407 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	408 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	409
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	410 void ConcurrentG1Refine::clear_and_record_card_counts() {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	411 if (G1ConcRSLogCacheSize == 0) return;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	412
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	413 #ifndef PRODUCT
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	414 double start = os::elapsedTime();
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	415 #endif
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	416
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	417 if (_expand_card_counts) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	418 expand_card_count_cache();
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	419 _expand_card_counts = false;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	420 // Only need to clear the epochs.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	421 Copy::fill_to_bytes(&_card_epochs[0], _n_card_counts * sizeof(CardEpochCacheEntry));
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	422 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	423
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	424 int this_epoch = (int) _n_periods;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	425 assert((this_epoch+1) <= max_jint, "to many periods");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	426 // Update epoch
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	427 _n_periods++;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	428
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	429 #ifndef PRODUCT
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	430 double elapsed = os::elapsedTime() - start;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	431 _g1h->g1_policy()->record_cc_clear_time(elapsed * 1000.0);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	432 #endif
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	433 }
1019 035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	434
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	435 void ConcurrentG1Refine::print_worker_threads_on(outputStream* st) const {
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	436 for (int i = 0; i < _n_threads; ++i) {
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	437 _threads[i]->print_on(st);
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	438 st->cr();
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	439 }
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	440 }

Mercurial > hg > graal-compiler

annotate src/share/vm/gc_implementation/g1/concurrentG1Refine.cpp @ 1974:fd1d227ef1b9