truffle: src/share/vm/gc_implementation/g1/concurrentG1Refine.cpp annotate

annotate src/share/vm/gc_implementation/g1/concurrentG1Refine.cpp @ 1666:5cbac8938c4c

6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 Summary: During concurrent refinment, filter cards in young regions after it has been determined that the region has been allocated from and the young type of the region has been set. Reviewed-by: iveresov, tonyp, jcoomes

author	johnc
date	Mon, 19 Jul 2010 11:06:34 -0700
parents	c18cbe5936b8
children	2d160770d2e5

rev	line source
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	1 /*
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	2 * Copyright (c) 2001, 2010, Oracle and/or its affiliates. All rights reserved.
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	4 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	5 * This code is free software; you can redistribute it and/or modify it
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	6 * under the terms of the GNU General Public License version 2 only, as
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	7 * published by the Free Software Foundation.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	8 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	9 * This code is distributed in the hope that it will be useful, but WITHOUT
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	10 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	11 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	12 * version 2 for more details (a copy is included in the LICENSE file that
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	13 * accompanied this code).
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	14 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	15 * You should have received a copy of the GNU General Public License version
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	16 * 2 along with this work; if not, write to the Free Software Foundation,
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	17 * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	18 *
1552 c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1282 diff changeset	19 * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1282 diff changeset	20 * or visit www.oracle.com if you need additional information or have any
c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1282 diff changeset	21 * questions.
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	22 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	23 */
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	24
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	25 #include "incls/_precompiled.incl"
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	26 #include "incls/_concurrentG1Refine.cpp.incl"
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	27
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	28 // Possible sizes for the card counts cache: odd primes that roughly double in size.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	29 // (See jvmtiTagMap.cpp).
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	30 int ConcurrentG1Refine::_cc_cache_sizes[] = {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	31 16381, 32771, 76831, 150001, 307261,
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	32 614563, 1228891, 2457733, 4915219, 9830479,
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	33 19660831, 39321619, 78643219, 157286461, -1
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	34 };
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	35
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	36 ConcurrentG1Refine::ConcurrentG1Refine() :
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	37 _card_counts(NULL), _card_epochs(NULL),
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	38 _n_card_counts(0), _max_n_card_counts(0),
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	39 _cache_size_index(0), _expand_card_counts(false),
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	40 _hot_cache(NULL),
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	41 _def_use_cache(false), _use_cache(false),
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	42 _n_periods(0),
794 315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	43 _threads(NULL), _n_threads(0)
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	44 {
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	45
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	46 // Ergomonically select initial concurrent refinement parameters
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	47 if (FLAG_IS_DEFAULT(G1ConcRefinementGreenZone)) {
b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	48 FLAG_SET_DEFAULT(G1ConcRefinementGreenZone, MAX2<int>(ParallelGCThreads, 1));
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	49 }
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	50 set_green_zone(G1ConcRefinementGreenZone);
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	51
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	52 if (FLAG_IS_DEFAULT(G1ConcRefinementYellowZone)) {
b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	53 FLAG_SET_DEFAULT(G1ConcRefinementYellowZone, green_zone() * 3);
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	54 }
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	55 set_yellow_zone(MAX2<int>(G1ConcRefinementYellowZone, green_zone()));
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	56
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	57 if (FLAG_IS_DEFAULT(G1ConcRefinementRedZone)) {
b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	58 FLAG_SET_DEFAULT(G1ConcRefinementRedZone, yellow_zone() * 2);
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	59 }
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	60 set_red_zone(MAX2<int>(G1ConcRefinementRedZone, yellow_zone()));
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	61 _n_worker_threads = thread_num();
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	62 // We need one extra thread to do the young gen rset size sampling.
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	63 _n_threads = _n_worker_threads + 1;
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	64 reset_threshold_step();
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	65
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	66 _threads = NEW_C_HEAP_ARRAY(ConcurrentG1RefineThread*, _n_threads);
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	67 int worker_id_offset = (int)DirtyCardQueueSet::num_par_ids();
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	68 ConcurrentG1RefineThread *next = NULL;
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	69 for (int i = _n_threads - 1; i >= 0; i--) {
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	70 ConcurrentG1RefineThread* t = new ConcurrentG1RefineThread(this, next, worker_id_offset, i);
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	71 assert(t != NULL, "Conc refine should have been created");
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	72 assert(t->cg1r() == this, "Conc refine thread should refer to this");
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	73 _threads[i] = t;
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	74 next = t;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	75 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	76 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	77
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	78 void ConcurrentG1Refine::reset_threshold_step() {
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	79 if (FLAG_IS_DEFAULT(G1ConcRefinementThresholdStep)) {
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	80 _thread_threshold_step = (yellow_zone() - green_zone()) / (worker_thread_num() + 1);
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	81 } else {
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	82 _thread_threshold_step = G1ConcRefinementThresholdStep;
795 215f81b4d9b3 6841831: G1: assert(contains_reference(from),"We just added it!") fires iveresov parents: 794 diff changeset	83 }
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	84 }
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	85
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	86 int ConcurrentG1Refine::thread_num() {
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	87 return MAX2<int>((G1ConcRefinementThreads > 0) ? G1ConcRefinementThreads : ParallelGCThreads, 1);
795 215f81b4d9b3 6841831: G1: assert(contains_reference(from),"We just added it!") fires iveresov parents: 794 diff changeset	88 }
215f81b4d9b3 6841831: G1: assert(contains_reference(from),"We just added it!") fires iveresov parents: 794 diff changeset	89
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	90 void ConcurrentG1Refine::init() {
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	91 if (G1ConcRSLogCacheSize > 0) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	92 _g1h = G1CollectedHeap::heap();
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	93 _max_n_card_counts =
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	94 (unsigned) (_g1h->g1_reserved_obj_bytes() >> CardTableModRefBS::card_shift);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	95
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	96 size_t max_card_num = ((size_t)1 << (sizeof(unsigned)*BitsPerByte-1)) - 1;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	97 guarantee(_max_n_card_counts < max_card_num, "card_num representation");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	98
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	99 int desired = _max_n_card_counts / InitialCacheFraction;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	100 for (_cache_size_index = 0;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	101 _cc_cache_sizes[_cache_size_index] >= 0; _cache_size_index++) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	102 if (_cc_cache_sizes[_cache_size_index] >= desired) break;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	103 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	104 _cache_size_index = MAX2(0, (_cache_size_index - 1));
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	105
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	106 int initial_size = _cc_cache_sizes[_cache_size_index];
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	107 if (initial_size < 0) initial_size = _max_n_card_counts;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	108
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	109 // Make sure we don't go bigger than we will ever need
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	110 _n_card_counts = MIN2((unsigned) initial_size, _max_n_card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	111
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	112 _card_counts = NEW_C_HEAP_ARRAY(CardCountCacheEntry, _n_card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	113 _card_epochs = NEW_C_HEAP_ARRAY(CardEpochCacheEntry, _n_card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	114
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	115 Copy::fill_to_bytes(&_card_counts[0],
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	116 _n_card_counts * sizeof(CardCountCacheEntry));
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	117 Copy::fill_to_bytes(&_card_epochs[0], _n_card_counts * sizeof(CardEpochCacheEntry));
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	118
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	119 ModRefBarrierSet* bs = _g1h->mr_bs();
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	120 guarantee(bs->is_a(BarrierSet::CardTableModRef), "Precondition");
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	121 _ct_bs = (CardTableModRefBS*)bs;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	122 _ct_bot = _ct_bs->byte_for_const(_g1h->reserved_region().start());
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	123
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	124 _def_use_cache = true;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	125 _use_cache = true;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	126 _hot_cache_size = (1 << G1ConcRSLogCacheSize);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	127 _hot_cache = NEW_C_HEAP_ARRAY(jbyte*, _hot_cache_size);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	128 _n_hot = 0;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	129 _hot_cache_idx = 0;
889 15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	130
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	131 // For refining the cards in the hot cache in parallel
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	132 int n_workers = (ParallelGCThreads > 0 ?
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	133 _g1h->workers()->total_workers() : 1);
889 15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	134 _hot_cache_par_chunk_size = MAX2(1, _hot_cache_size / n_workers);
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	135 _hot_cache_par_claimed_idx = 0;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	136 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	137 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	138
794 315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	139 void ConcurrentG1Refine::stop() {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	140 if (_threads != NULL) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	141 for (int i = 0; i < _n_threads; i++) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	142 _threads[i]->stop();
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	143 }
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	144 }
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	145 }
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	146
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	147 void ConcurrentG1Refine::reinitialize_threads() {
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	148 reset_threshold_step();
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	149 if (_threads != NULL) {
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	150 for (int i = 0; i < _n_threads; i++) {
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	151 _threads[i]->initialize();
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	152 }
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	153 }
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	154 }
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	155
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	156 ConcurrentG1Refine::~ConcurrentG1Refine() {
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	157 if (G1ConcRSLogCacheSize > 0) {
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	158 assert(_card_counts != NULL, "Logic");
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	159 FREE_C_HEAP_ARRAY(CardCountCacheEntry, _card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	160 assert(_card_epochs != NULL, "Logic");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	161 FREE_C_HEAP_ARRAY(CardEpochCacheEntry, _card_epochs);
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	162 assert(_hot_cache != NULL, "Logic");
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	163 FREE_C_HEAP_ARRAY(jbyte*, _hot_cache);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	164 }
794 315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	165 if (_threads != NULL) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	166 for (int i = 0; i < _n_threads; i++) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	167 delete _threads[i];
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	168 }
799 f89cf529c3c7 6849122: G1: Typo introduced during implementation of the parallel refinement iveresov parents: 795 diff changeset	169 FREE_C_HEAP_ARRAY(ConcurrentG1RefineThread*, _threads);
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	170 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	171 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	172
794 315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	173 void ConcurrentG1Refine::threads_do(ThreadClosure *tc) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	174 if (_threads != NULL) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	175 for (int i = 0; i < _n_threads; i++) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	176 tc->do_thread(_threads[i]);
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	177 }
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	178 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	179 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	180
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	181 bool ConcurrentG1Refine::is_young_card(jbyte* card_ptr) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	182 HeapWord* start = _ct_bs->addr_for(card_ptr);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	183 HeapRegion* r = _g1h->heap_region_containing(start);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	184 if (r != NULL && r->is_young()) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	185 return true;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	186 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	187 // This card is not associated with a heap region
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	188 // so can't be young.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	189 return false;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	190 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	191
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	192 jbyte* ConcurrentG1Refine::add_card_count(jbyte* card_ptr, int* count, bool* defer) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	193 unsigned new_card_num = ptr_2_card_num(card_ptr);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	194 unsigned bucket = hash(new_card_num);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	195 assert(0 <= bucket && bucket < _n_card_counts, "Bounds");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	196
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	197 CardCountCacheEntry* count_ptr = &_card_counts[bucket];
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	198 CardEpochCacheEntry* epoch_ptr = &_card_epochs[bucket];
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	199
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	200 // We have to construct a new entry if we haven't updated the counts
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	201 // during the current period, or if the count was updated for a
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	202 // different card number.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	203 unsigned int new_epoch = (unsigned int) _n_periods;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	204 julong new_epoch_entry = make_epoch_entry(new_card_num, new_epoch);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	205
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	206 while (true) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	207 // Fetch the previous epoch value
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	208 julong prev_epoch_entry = epoch_ptr->_value;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	209 julong cas_res;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	210
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	211 if (extract_epoch(prev_epoch_entry) != new_epoch) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	212 // This entry has not yet been updated during this period.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	213 // Note: we update the epoch value atomically to ensure
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	214 // that there is only one winner that updates the cached
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	215 // card_ptr value even though all the refine threads share
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	216 // the same epoch value.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	217
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	218 cas_res = (julong) Atomic::cmpxchg((jlong) new_epoch_entry,
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	219 (volatile jlong*)&epoch_ptr->_value,
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	220 (jlong) prev_epoch_entry);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	221
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	222 if (cas_res == prev_epoch_entry) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	223 // We have successfully won the race to update the
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	224 // epoch and card_num value. Make it look like the
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	225 // count and eviction count were previously cleared.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	226 count_ptr->_count = 1;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	227 count_ptr->_evict_count = 0;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	228 *count = 0;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	229 // We can defer the processing of card_ptr
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	230 *defer = true;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	231 return card_ptr;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	232 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	233 // We did not win the race to update the epoch field, so some other
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	234 // thread must have done it. The value that gets returned by CAS
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	235 // should be the new epoch value.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	236 assert(extract_epoch(cas_res) == new_epoch, "unexpected epoch");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	237 // We could 'continue' here or just re-read the previous epoch value
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	238 prev_epoch_entry = epoch_ptr->_value;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	239 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	240
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	241 // The epoch entry for card_ptr has been updated during this period.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	242 unsigned old_card_num = extract_card_num(prev_epoch_entry);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	243
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	244 // The card count that will be returned to caller
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	245 *count = count_ptr->_count;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	246
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	247 // Are we updating the count for the same card?
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	248 if (new_card_num == old_card_num) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	249 // Same card - just update the count. We could have more than one
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	250 // thread racing to update count for the current card. It should be
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	251 // OK not to use a CAS as the only penalty should be some missed
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	252 // increments of the count which delays identifying the card as "hot".
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	253
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	254 if (*count < max_jubyte) count_ptr->_count++;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	255 // We can defer the processing of card_ptr
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	256 *defer = true;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	257 return card_ptr;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	258 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	259
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	260 // Different card - evict old card info
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	261 if (count_ptr->_evict_count < max_jubyte) count_ptr->_evict_count++;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	262 if (count_ptr->_evict_count > G1CardCountCacheExpandThreshold) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	263 // Trigger a resize the next time we clear
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	264 _expand_card_counts = true;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	265 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	266
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	267 cas_res = (julong) Atomic::cmpxchg((jlong) new_epoch_entry,
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	268 (volatile jlong*)&epoch_ptr->_value,
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	269 (jlong) prev_epoch_entry);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	270
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	271 if (cas_res == prev_epoch_entry) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	272 // We successfully updated the card num value in the epoch entry
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	273 count_ptr->_count = 0; // initialize counter for new card num
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	274 jbyte* old_card_ptr = card_num_2_ptr(old_card_num);
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	275
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	276 // Even though the region containg the card at old_card_num was not
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	277 // in the young list when old_card_num was recorded in the epoch
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	278 // cache it could have been added to the free list and subsequently
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	279 // added to the young list in the intervening time. See CR 6817995.
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	280 // We do not deal with this case here - it will be handled in
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	281 // HeapRegion::oops_on_card_seq_iterate_careful after it has been
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	282 // determined that the region containing the card has been allocated
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	283 // to, and it's safe to check the young type of the region.
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	284
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	285 // We do not want to defer processing of card_ptr in this case
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	286 // (we need to refine old_card_ptr and card_ptr)
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	287 *defer = false;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	288 return old_card_ptr;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	289 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	290 // Someone else beat us - try again.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	291 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	292 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	293
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	294 jbyte* ConcurrentG1Refine::cache_insert(jbyte* card_ptr, bool* defer) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	295 int count;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	296 jbyte* cached_ptr = add_card_count(card_ptr, &count, defer);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	297 assert(cached_ptr != NULL, "bad cached card ptr");
1246 deada8912c54 6914402: G1: assert(!is_young_card(cached_ptr),"shouldn't get a card in young region") johnc parents: 1111 diff changeset	298
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	299 // We've just inserted a card pointer into the card count cache
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	300 // and got back the card that we just inserted or (evicted) the
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	301 // previous contents of that count slot.
1246 deada8912c54 6914402: G1: assert(!is_young_card(cached_ptr),"shouldn't get a card in young region") johnc parents: 1111 diff changeset	302
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	303 // The card we got back could be in a young region. When the
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	304 // returned card (if evicted) was originally inserted, we had
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	305 // determined that its containing region was not young. However
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	306 // it is possible for the region to be freed during a cleanup
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	307 // pause, then reallocated and tagged as young which will result
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	308 // in the returned card residing in a young region.
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	309 //
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	310 // We do not deal with this case here - the change from non-young
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	311 // to young could be observed at any time - it will be handled in
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	312 // HeapRegion::oops_on_card_seq_iterate_careful after it has been
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	313 // determined that the region containing the card has been allocated
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	314 // to.
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	315
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	316 // The card pointer we obtained from card count cache is not hot
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	317 // so do not store it in the cache; return it for immediate
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	318 // refining.
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	319 if (count < G1ConcRSHotCardLimit) {
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	320 return cached_ptr;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	321 }
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	322
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	323 // Otherwise, the pointer we got from the _card_counts cache is hot.
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	324 jbyte* res = NULL;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	325 MutexLockerEx x(HotCardCache_lock, Mutex::_no_safepoint_check_flag);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	326 if (_n_hot == _hot_cache_size) {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	327 res = _hot_cache[_hot_cache_idx];
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	328 _n_hot--;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	329 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	330 // Now _n_hot < _hot_cache_size, and we can insert at _hot_cache_idx.
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	331 _hot_cache[_hot_cache_idx] = cached_ptr;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	332 _hot_cache_idx++;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	333 if (_hot_cache_idx == _hot_cache_size) _hot_cache_idx = 0;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	334 _n_hot++;
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	335
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	336 // The card obtained from the hot card cache could be in a young
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	337 // region. See above on how this can happen.
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	338
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	339 return res;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	340 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	341
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	342 void ConcurrentG1Refine::clean_up_cache(int worker_i, G1RemSet* g1rs) {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	343 assert(!use_cache(), "cache should be disabled");
889 15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	344 int start_idx;
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	345
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	346 while ((start_idx = _hot_cache_par_claimed_idx) < _n_hot) { // read once
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	347 int end_idx = start_idx + _hot_cache_par_chunk_size;
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	348
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	349 if (start_idx ==
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	350 Atomic::cmpxchg(end_idx, &_hot_cache_par_claimed_idx, start_idx)) {
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	351 // The current worker has successfully claimed the chunk [start_idx..end_idx)
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	352 end_idx = MIN2(end_idx, _n_hot);
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	353 for (int i = start_idx; i < end_idx; i++) {
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	354 jbyte* entry = _hot_cache[i];
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	355 if (entry != NULL) {
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	356 g1rs->concurrentRefineOneCard(entry, worker_i);
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	357 }
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	358 }
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	359 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	360 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	361 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	362
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	363 void ConcurrentG1Refine::expand_card_count_cache() {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	364 if (_n_card_counts < _max_n_card_counts) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	365 int new_idx = _cache_size_index+1;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	366 int new_size = _cc_cache_sizes[new_idx];
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	367 if (new_size < 0) new_size = _max_n_card_counts;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	368
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	369 // Make sure we don't go bigger than we will ever need
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	370 new_size = MIN2((unsigned) new_size, _max_n_card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	371
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	372 // Expand the card count and card epoch tables
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	373 if (new_size > (int)_n_card_counts) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	374 // We can just free and allocate a new array as we're
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	375 // not interested in preserving the contents
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	376 assert(_card_counts != NULL, "Logic!");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	377 assert(_card_epochs != NULL, "Logic!");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	378 FREE_C_HEAP_ARRAY(CardCountCacheEntry, _card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	379 FREE_C_HEAP_ARRAY(CardEpochCacheEntry, _card_epochs);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	380 _n_card_counts = new_size;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	381 _card_counts = NEW_C_HEAP_ARRAY(CardCountCacheEntry, _n_card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	382 _card_epochs = NEW_C_HEAP_ARRAY(CardEpochCacheEntry, _n_card_counts);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	383 _cache_size_index = new_idx;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	384 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	385 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	386 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	387
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	388 void ConcurrentG1Refine::clear_and_record_card_counts() {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	389 if (G1ConcRSLogCacheSize == 0) return;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	390
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	391 #ifndef PRODUCT
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	392 double start = os::elapsedTime();
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	393 #endif
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	394
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	395 if (_expand_card_counts) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	396 expand_card_count_cache();
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	397 _expand_card_counts = false;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	398 // Only need to clear the epochs.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	399 Copy::fill_to_bytes(&_card_epochs[0], _n_card_counts * sizeof(CardEpochCacheEntry));
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	400 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	401
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	402 int this_epoch = (int) _n_periods;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	403 assert((this_epoch+1) <= max_jint, "to many periods");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	404 // Update epoch
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	405 _n_periods++;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	406
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	407 #ifndef PRODUCT
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	408 double elapsed = os::elapsedTime() - start;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	409 _g1h->g1_policy()->record_cc_clear_time(elapsed * 1000.0);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	410 #endif
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	411 }
1019 035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	412
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	413 void ConcurrentG1Refine::print_worker_threads_on(outputStream* st) const {
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	414 for (int i = 0; i < _n_threads; ++i) {
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	415 _threads[i]->print_on(st);
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	416 st->cr();
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	417 }
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	418 }

Mercurial > hg > truffle

annotate src/share/vm/gc_implementation/g1/concurrentG1Refine.cpp @ 1666:5cbac8938c4c