truffle: src/share/vm/gc_implementation/g1/concurrentG1Refine.cpp annotate

annotate src/share/vm/gc_implementation/g1/concurrentG1Refine.cpp @ 1972:f95d63e2154a

6989984: Use standard include model for Hospot Summary: Replaced MakeDeps and the includeDB files with more standardized solutions. Reviewed-by: coleenp, kvn, kamg

author	stefank
date	Tue, 23 Nov 2010 13:22:55 -0800
parents	2d160770d2e5
children	c33825b68624

rev	line source
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	1 /*
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	2 * Copyright (c) 2001, 2010, Oracle and/or its affiliates. All rights reserved.
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	4 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	5 * This code is free software; you can redistribute it and/or modify it
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	6 * under the terms of the GNU General Public License version 2 only, as
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	7 * published by the Free Software Foundation.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	8 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	9 * This code is distributed in the hope that it will be useful, but WITHOUT
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	10 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	11 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	12 * version 2 for more details (a copy is included in the LICENSE file that
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	13 * accompanied this code).
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	14 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	15 * You should have received a copy of the GNU General Public License version
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	16 * 2 along with this work; if not, write to the Free Software Foundation,
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	17 * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	18 *
1552 c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1282 diff changeset	19 * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1282 diff changeset	20 * or visit www.oracle.com if you need additional information or have any
c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1282 diff changeset	21 * questions.
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	22 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	23 */
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	24
1972 f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	25 #include "precompiled.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	26 #include "gc_implementation/g1/concurrentG1Refine.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	27 #include "gc_implementation/g1/concurrentG1RefineThread.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	28 #include "gc_implementation/g1/g1CollectedHeap.inline.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	29 #include "gc_implementation/g1/g1CollectorPolicy.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	30 #include "gc_implementation/g1/g1RemSet.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	31 #include "gc_implementation/g1/heapRegionSeq.inline.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	32 #include "memory/space.inline.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	33 #include "runtime/atomic.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	34 #include "utilities/copy.hpp"
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	35
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	36 // Possible sizes for the card counts cache: odd primes that roughly double in size.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	37 // (See jvmtiTagMap.cpp).
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	38 int ConcurrentG1Refine::_cc_cache_sizes[] = {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	39 16381, 32771, 76831, 150001, 307261,
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	40 614563, 1228891, 2457733, 4915219, 9830479,
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	41 19660831, 39321619, 78643219, 157286461, -1
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	42 };
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	43
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	44 ConcurrentG1Refine::ConcurrentG1Refine() :
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	45 _card_counts(NULL), _card_epochs(NULL),
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	46 _n_card_counts(0), _max_n_card_counts(0),
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	47 _cache_size_index(0), _expand_card_counts(false),
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	48 _hot_cache(NULL),
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	49 _def_use_cache(false), _use_cache(false),
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	50 _n_periods(0),
794 315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	51 _threads(NULL), _n_threads(0)
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	52 {
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	53
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	54 // Ergomonically select initial concurrent refinement parameters
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	55 if (FLAG_IS_DEFAULT(G1ConcRefinementGreenZone)) {
b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	56 FLAG_SET_DEFAULT(G1ConcRefinementGreenZone, MAX2<int>(ParallelGCThreads, 1));
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	57 }
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	58 set_green_zone(G1ConcRefinementGreenZone);
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	59
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	60 if (FLAG_IS_DEFAULT(G1ConcRefinementYellowZone)) {
b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	61 FLAG_SET_DEFAULT(G1ConcRefinementYellowZone, green_zone() * 3);
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	62 }
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	63 set_yellow_zone(MAX2<int>(G1ConcRefinementYellowZone, green_zone()));
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	64
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	65 if (FLAG_IS_DEFAULT(G1ConcRefinementRedZone)) {
b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	66 FLAG_SET_DEFAULT(G1ConcRefinementRedZone, yellow_zone() * 2);
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	67 }
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	68 set_red_zone(MAX2<int>(G1ConcRefinementRedZone, yellow_zone()));
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	69 _n_worker_threads = thread_num();
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	70 // We need one extra thread to do the young gen rset size sampling.
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	71 _n_threads = _n_worker_threads + 1;
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	72 reset_threshold_step();
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	73
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	74 _threads = NEW_C_HEAP_ARRAY(ConcurrentG1RefineThread*, _n_threads);
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	75 int worker_id_offset = (int)DirtyCardQueueSet::num_par_ids();
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	76 ConcurrentG1RefineThread *next = NULL;
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	77 for (int i = _n_threads - 1; i >= 0; i--) {
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	78 ConcurrentG1RefineThread* t = new ConcurrentG1RefineThread(this, next, worker_id_offset, i);
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	79 assert(t != NULL, "Conc refine should have been created");
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	80 assert(t->cg1r() == this, "Conc refine thread should refer to this");
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	81 _threads[i] = t;
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	82 next = t;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	83 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	84 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	85
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	86 void ConcurrentG1Refine::reset_threshold_step() {
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	87 if (FLAG_IS_DEFAULT(G1ConcRefinementThresholdStep)) {
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	88 _thread_threshold_step = (yellow_zone() - green_zone()) / (worker_thread_num() + 1);
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	89 } else {
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	90 _thread_threshold_step = G1ConcRefinementThresholdStep;
795 215f81b4d9b3 6841831: G1: assert(contains_reference(from),"We just added it!") fires iveresov parents: 794 diff changeset	91 }
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	92 }
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	93
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	94 int ConcurrentG1Refine::thread_num() {
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	95 return MAX2<int>((G1ConcRefinementThreads > 0) ? G1ConcRefinementThreads : ParallelGCThreads, 1);
795 215f81b4d9b3 6841831: G1: assert(contains_reference(from),"We just added it!") fires iveresov parents: 794 diff changeset	96 }
215f81b4d9b3 6841831: G1: assert(contains_reference(from),"We just added it!") fires iveresov parents: 794 diff changeset	97
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	98 void ConcurrentG1Refine::init() {
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	99 if (G1ConcRSLogCacheSize > 0) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	100 _g1h = G1CollectedHeap::heap();
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	101 _max_n_card_counts =
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	102 (unsigned) (_g1h->g1_reserved_obj_bytes() >> CardTableModRefBS::card_shift);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	103
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	104 size_t max_card_num = ((size_t)1 << (sizeof(unsigned)*BitsPerByte-1)) - 1;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	105 guarantee(_max_n_card_counts < max_card_num, "card_num representation");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	106
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	107 int desired = _max_n_card_counts / InitialCacheFraction;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	108 for (_cache_size_index = 0;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	109 _cc_cache_sizes[_cache_size_index] >= 0; _cache_size_index++) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	110 if (_cc_cache_sizes[_cache_size_index] >= desired) break;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	111 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	112 _cache_size_index = MAX2(0, (_cache_size_index - 1));
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	113
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	114 int initial_size = _cc_cache_sizes[_cache_size_index];
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	115 if (initial_size < 0) initial_size = _max_n_card_counts;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	116
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	117 // Make sure we don't go bigger than we will ever need
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	118 _n_card_counts = MIN2((unsigned) initial_size, _max_n_card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	119
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	120 _card_counts = NEW_C_HEAP_ARRAY(CardCountCacheEntry, _n_card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	121 _card_epochs = NEW_C_HEAP_ARRAY(CardEpochCacheEntry, _n_card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	122
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	123 Copy::fill_to_bytes(&_card_counts[0],
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	124 _n_card_counts * sizeof(CardCountCacheEntry));
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	125 Copy::fill_to_bytes(&_card_epochs[0], _n_card_counts * sizeof(CardEpochCacheEntry));
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	126
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	127 ModRefBarrierSet* bs = _g1h->mr_bs();
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	128 guarantee(bs->is_a(BarrierSet::CardTableModRef), "Precondition");
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	129 _ct_bs = (CardTableModRefBS*)bs;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	130 _ct_bot = _ct_bs->byte_for_const(_g1h->reserved_region().start());
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	131
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	132 _def_use_cache = true;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	133 _use_cache = true;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	134 _hot_cache_size = (1 << G1ConcRSLogCacheSize);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	135 _hot_cache = NEW_C_HEAP_ARRAY(jbyte*, _hot_cache_size);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	136 _n_hot = 0;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	137 _hot_cache_idx = 0;
889 15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	138
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	139 // For refining the cards in the hot cache in parallel
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	140 int n_workers = (ParallelGCThreads > 0 ?
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	141 _g1h->workers()->total_workers() : 1);
889 15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	142 _hot_cache_par_chunk_size = MAX2(1, _hot_cache_size / n_workers);
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	143 _hot_cache_par_claimed_idx = 0;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	144 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	145 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	146
794 315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	147 void ConcurrentG1Refine::stop() {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	148 if (_threads != NULL) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	149 for (int i = 0; i < _n_threads; i++) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	150 _threads[i]->stop();
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	151 }
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	152 }
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	153 }
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	154
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	155 void ConcurrentG1Refine::reinitialize_threads() {
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	156 reset_threshold_step();
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	157 if (_threads != NULL) {
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	158 for (int i = 0; i < _n_threads; i++) {
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	159 _threads[i]->initialize();
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	160 }
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	161 }
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	162 }
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	163
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	164 ConcurrentG1Refine::~ConcurrentG1Refine() {
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	165 if (G1ConcRSLogCacheSize > 0) {
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	166 assert(_card_counts != NULL, "Logic");
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	167 FREE_C_HEAP_ARRAY(CardCountCacheEntry, _card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	168 assert(_card_epochs != NULL, "Logic");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	169 FREE_C_HEAP_ARRAY(CardEpochCacheEntry, _card_epochs);
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	170 assert(_hot_cache != NULL, "Logic");
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	171 FREE_C_HEAP_ARRAY(jbyte*, _hot_cache);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	172 }
794 315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	173 if (_threads != NULL) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	174 for (int i = 0; i < _n_threads; i++) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	175 delete _threads[i];
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	176 }
799 f89cf529c3c7 6849122: G1: Typo introduced during implementation of the parallel refinement iveresov parents: 795 diff changeset	177 FREE_C_HEAP_ARRAY(ConcurrentG1RefineThread*, _threads);
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	178 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	179 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	180
794 315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	181 void ConcurrentG1Refine::threads_do(ThreadClosure *tc) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	182 if (_threads != NULL) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	183 for (int i = 0; i < _n_threads; i++) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	184 tc->do_thread(_threads[i]);
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	185 }
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	186 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	187 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	188
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	189 bool ConcurrentG1Refine::is_young_card(jbyte* card_ptr) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	190 HeapWord* start = _ct_bs->addr_for(card_ptr);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	191 HeapRegion* r = _g1h->heap_region_containing(start);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	192 if (r != NULL && r->is_young()) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	193 return true;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	194 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	195 // This card is not associated with a heap region
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	196 // so can't be young.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	197 return false;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	198 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	199
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	200 jbyte* ConcurrentG1Refine::add_card_count(jbyte* card_ptr, int* count, bool* defer) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	201 unsigned new_card_num = ptr_2_card_num(card_ptr);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	202 unsigned bucket = hash(new_card_num);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	203 assert(0 <= bucket && bucket < _n_card_counts, "Bounds");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	204
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	205 CardCountCacheEntry* count_ptr = &_card_counts[bucket];
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	206 CardEpochCacheEntry* epoch_ptr = &_card_epochs[bucket];
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	207
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	208 // We have to construct a new entry if we haven't updated the counts
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	209 // during the current period, or if the count was updated for a
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	210 // different card number.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	211 unsigned int new_epoch = (unsigned int) _n_periods;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	212 julong new_epoch_entry = make_epoch_entry(new_card_num, new_epoch);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	213
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	214 while (true) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	215 // Fetch the previous epoch value
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	216 julong prev_epoch_entry = epoch_ptr->_value;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	217 julong cas_res;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	218
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	219 if (extract_epoch(prev_epoch_entry) != new_epoch) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	220 // This entry has not yet been updated during this period.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	221 // Note: we update the epoch value atomically to ensure
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	222 // that there is only one winner that updates the cached
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	223 // card_ptr value even though all the refine threads share
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	224 // the same epoch value.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	225
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	226 cas_res = (julong) Atomic::cmpxchg((jlong) new_epoch_entry,
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	227 (volatile jlong*)&epoch_ptr->_value,
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	228 (jlong) prev_epoch_entry);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	229
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	230 if (cas_res == prev_epoch_entry) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	231 // We have successfully won the race to update the
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	232 // epoch and card_num value. Make it look like the
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	233 // count and eviction count were previously cleared.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	234 count_ptr->_count = 1;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	235 count_ptr->_evict_count = 0;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	236 *count = 0;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	237 // We can defer the processing of card_ptr
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	238 *defer = true;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	239 return card_ptr;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	240 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	241 // We did not win the race to update the epoch field, so some other
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	242 // thread must have done it. The value that gets returned by CAS
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	243 // should be the new epoch value.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	244 assert(extract_epoch(cas_res) == new_epoch, "unexpected epoch");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	245 // We could 'continue' here or just re-read the previous epoch value
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	246 prev_epoch_entry = epoch_ptr->_value;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	247 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	248
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	249 // The epoch entry for card_ptr has been updated during this period.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	250 unsigned old_card_num = extract_card_num(prev_epoch_entry);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	251
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	252 // The card count that will be returned to caller
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	253 *count = count_ptr->_count;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	254
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	255 // Are we updating the count for the same card?
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	256 if (new_card_num == old_card_num) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	257 // Same card - just update the count. We could have more than one
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	258 // thread racing to update count for the current card. It should be
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	259 // OK not to use a CAS as the only penalty should be some missed
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	260 // increments of the count which delays identifying the card as "hot".
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	261
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	262 if (*count < max_jubyte) count_ptr->_count++;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	263 // We can defer the processing of card_ptr
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	264 *defer = true;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	265 return card_ptr;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	266 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	267
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	268 // Different card - evict old card info
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	269 if (count_ptr->_evict_count < max_jubyte) count_ptr->_evict_count++;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	270 if (count_ptr->_evict_count > G1CardCountCacheExpandThreshold) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	271 // Trigger a resize the next time we clear
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	272 _expand_card_counts = true;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	273 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	274
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	275 cas_res = (julong) Atomic::cmpxchg((jlong) new_epoch_entry,
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	276 (volatile jlong*)&epoch_ptr->_value,
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	277 (jlong) prev_epoch_entry);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	278
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	279 if (cas_res == prev_epoch_entry) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	280 // We successfully updated the card num value in the epoch entry
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	281 count_ptr->_count = 0; // initialize counter for new card num
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	282 jbyte* old_card_ptr = card_num_2_ptr(old_card_num);
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	283
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	284 // Even though the region containg the card at old_card_num was not
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	285 // in the young list when old_card_num was recorded in the epoch
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	286 // cache it could have been added to the free list and subsequently
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	287 // added to the young list in the intervening time. See CR 6817995.
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	288 // We do not deal with this case here - it will be handled in
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	289 // HeapRegion::oops_on_card_seq_iterate_careful after it has been
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	290 // determined that the region containing the card has been allocated
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	291 // to, and it's safe to check the young type of the region.
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	292
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	293 // We do not want to defer processing of card_ptr in this case
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	294 // (we need to refine old_card_ptr and card_ptr)
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	295 *defer = false;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	296 return old_card_ptr;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	297 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	298 // Someone else beat us - try again.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	299 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	300 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	301
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	302 jbyte* ConcurrentG1Refine::cache_insert(jbyte* card_ptr, bool* defer) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	303 int count;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	304 jbyte* cached_ptr = add_card_count(card_ptr, &count, defer);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	305 assert(cached_ptr != NULL, "bad cached card ptr");
1246 deada8912c54 6914402: G1: assert(!is_young_card(cached_ptr),"shouldn't get a card in young region") johnc parents: 1111 diff changeset	306
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	307 // We've just inserted a card pointer into the card count cache
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	308 // and got back the card that we just inserted or (evicted) the
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	309 // previous contents of that count slot.
1246 deada8912c54 6914402: G1: assert(!is_young_card(cached_ptr),"shouldn't get a card in young region") johnc parents: 1111 diff changeset	310
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	311 // The card we got back could be in a young region. When the
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	312 // returned card (if evicted) was originally inserted, we had
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	313 // determined that its containing region was not young. However
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	314 // it is possible for the region to be freed during a cleanup
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	315 // pause, then reallocated and tagged as young which will result
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	316 // in the returned card residing in a young region.
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	317 //
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	318 // We do not deal with this case here - the change from non-young
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	319 // to young could be observed at any time - it will be handled in
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	320 // HeapRegion::oops_on_card_seq_iterate_careful after it has been
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	321 // determined that the region containing the card has been allocated
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	322 // to.
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	323
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	324 // The card pointer we obtained from card count cache is not hot
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	325 // so do not store it in the cache; return it for immediate
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	326 // refining.
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	327 if (count < G1ConcRSHotCardLimit) {
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	328 return cached_ptr;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	329 }
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	330
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	331 // Otherwise, the pointer we got from the _card_counts cache is hot.
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	332 jbyte* res = NULL;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	333 MutexLockerEx x(HotCardCache_lock, Mutex::_no_safepoint_check_flag);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	334 if (_n_hot == _hot_cache_size) {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	335 res = _hot_cache[_hot_cache_idx];
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	336 _n_hot--;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	337 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	338 // Now _n_hot < _hot_cache_size, and we can insert at _hot_cache_idx.
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	339 _hot_cache[_hot_cache_idx] = cached_ptr;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	340 _hot_cache_idx++;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	341 if (_hot_cache_idx == _hot_cache_size) _hot_cache_idx = 0;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	342 _n_hot++;
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	343
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	344 // The card obtained from the hot card cache could be in a young
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	345 // region. See above on how this can happen.
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	346
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	347 return res;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	348 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	349
1705 2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	350 void ConcurrentG1Refine::clean_up_cache(int worker_i,
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	351 G1RemSet* g1rs,
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	352 DirtyCardQueue* into_cset_dcq) {
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	353 assert(!use_cache(), "cache should be disabled");
889 15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	354 int start_idx;
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	355
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	356 while ((start_idx = _hot_cache_par_claimed_idx) < _n_hot) { // read once
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	357 int end_idx = start_idx + _hot_cache_par_chunk_size;
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	358
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	359 if (start_idx ==
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	360 Atomic::cmpxchg(end_idx, &_hot_cache_par_claimed_idx, start_idx)) {
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	361 // The current worker has successfully claimed the chunk [start_idx..end_idx)
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	362 end_idx = MIN2(end_idx, _n_hot);
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	363 for (int i = start_idx; i < end_idx; i++) {
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	364 jbyte* entry = _hot_cache[i];
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	365 if (entry != NULL) {
1705 2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	366 if (g1rs->concurrentRefineOneCard(entry, worker_i, true)) {
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	367 // 'entry' contains references that point into the current
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	368 // collection set. We need to record 'entry' in the DCQS
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	369 // that's used for that purpose.
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	370 //
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	371 // The only time we care about recording cards that contain
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	372 // references that point into the collection set is during
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	373 // RSet updating while within an evacuation pause.
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	374 // In this case worker_i should be the id of a GC worker thread
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	375 assert(SafepointSynchronize::is_at_safepoint(), "not during an evacuation pause");
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	376 assert(worker_i < (int) DirtyCardQueueSet::num_par_ids(), "incorrect worker id");
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	377 into_cset_dcq->enqueue(entry);
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	378 }
889 15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	379 }
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	380 }
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	381 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	382 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	383 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	384
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	385 void ConcurrentG1Refine::expand_card_count_cache() {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	386 if (_n_card_counts < _max_n_card_counts) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	387 int new_idx = _cache_size_index+1;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	388 int new_size = _cc_cache_sizes[new_idx];
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	389 if (new_size < 0) new_size = _max_n_card_counts;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	390
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	391 // Make sure we don't go bigger than we will ever need
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	392 new_size = MIN2((unsigned) new_size, _max_n_card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	393
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	394 // Expand the card count and card epoch tables
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	395 if (new_size > (int)_n_card_counts) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	396 // We can just free and allocate a new array as we're
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	397 // not interested in preserving the contents
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	398 assert(_card_counts != NULL, "Logic!");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	399 assert(_card_epochs != NULL, "Logic!");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	400 FREE_C_HEAP_ARRAY(CardCountCacheEntry, _card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	401 FREE_C_HEAP_ARRAY(CardEpochCacheEntry, _card_epochs);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	402 _n_card_counts = new_size;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	403 _card_counts = NEW_C_HEAP_ARRAY(CardCountCacheEntry, _n_card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	404 _card_epochs = NEW_C_HEAP_ARRAY(CardEpochCacheEntry, _n_card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	405 _cache_size_index = new_idx;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	406 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	407 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	408 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	409
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	410 void ConcurrentG1Refine::clear_and_record_card_counts() {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	411 if (G1ConcRSLogCacheSize == 0) return;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	412
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	413 #ifndef PRODUCT
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	414 double start = os::elapsedTime();
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	415 #endif
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	416
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	417 if (_expand_card_counts) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	418 expand_card_count_cache();
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	419 _expand_card_counts = false;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	420 // Only need to clear the epochs.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	421 Copy::fill_to_bytes(&_card_epochs[0], _n_card_counts * sizeof(CardEpochCacheEntry));
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	422 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	423
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	424 int this_epoch = (int) _n_periods;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	425 assert((this_epoch+1) <= max_jint, "to many periods");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	426 // Update epoch
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	427 _n_periods++;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	428
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	429 #ifndef PRODUCT
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	430 double elapsed = os::elapsedTime() - start;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	431 _g1h->g1_policy()->record_cc_clear_time(elapsed * 1000.0);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	432 #endif
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	433 }
1019 035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	434
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	435 void ConcurrentG1Refine::print_worker_threads_on(outputStream* st) const {
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	436 for (int i = 0; i < _n_threads; ++i) {
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	437 _threads[i]->print_on(st);
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	438 st->cr();
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	439 }
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	440 }

Mercurial > hg > truffle

annotate src/share/vm/gc_implementation/g1/concurrentG1Refine.cpp @ 1972:f95d63e2154a