truffle: src/share/vm/gc_implementation/g1/concurrentG1Refine.cpp annotate

annotate src/share/vm/gc_implementation/g1/concurrentG1Refine.cpp @ 8733:9def4075da6d

8008079: G1: Add nextObject routine to CMBitMapRO and replace nextWord Summary: Update the task local finger to the start of the next object when marking aborts, in order to avoid the redundant scanning of all 0's when the marking task restarts, if otherwise updating to the next word. In addition, reuse the routine nextObject() in routine iterate(). Reviewed-by: johnc, ysr Contributed-by: tamao <tao.mao@oracle.com>

author	tamao
date	Tue, 05 Mar 2013 15:36:56 -0800
parents	3a431b605145
children	194f52aa2f23

rev	line source
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	1 /*
6219 922993931b3d 7178361: G1: Make sure that PrintGC and PrintGCDetails use the same timing for the GC pause brutisso parents: 2434 diff changeset	2 * Copyright (c) 2001, 2012, Oracle and/or its affiliates. All rights reserved.
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	4 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	5 * This code is free software; you can redistribute it and/or modify it
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	6 * under the terms of the GNU General Public License version 2 only, as
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	7 * published by the Free Software Foundation.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	8 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	9 * This code is distributed in the hope that it will be useful, but WITHOUT
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	10 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	11 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	12 * version 2 for more details (a copy is included in the LICENSE file that
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	13 * accompanied this code).
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	14 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	15 * You should have received a copy of the GNU General Public License version
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	16 * 2 along with this work; if not, write to the Free Software Foundation,
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	17 * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	18 *
1552 c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1282 diff changeset	19 * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1282 diff changeset	20 * or visit www.oracle.com if you need additional information or have any
c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1282 diff changeset	21 * questions.
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	22 *
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	23 */
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	24
1972 f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	25 #include "precompiled.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	26 #include "gc_implementation/g1/concurrentG1Refine.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	27 #include "gc_implementation/g1/concurrentG1RefineThread.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	28 #include "gc_implementation/g1/g1CollectedHeap.inline.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	29 #include "gc_implementation/g1/g1CollectorPolicy.hpp"
6219 922993931b3d 7178361: G1: Make sure that PrintGC and PrintGCDetails use the same timing for the GC pause brutisso parents: 2434 diff changeset	30 #include "gc_implementation/g1/g1GCPhaseTimes.hpp"
1972 f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	31 #include "gc_implementation/g1/g1RemSet.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	32 #include "gc_implementation/g1/heapRegionSeq.inline.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	33 #include "memory/space.inline.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	34 #include "runtime/atomic.hpp"
2431 02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	35 #include "runtime/java.hpp"
1972 f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1705 diff changeset	36 #include "utilities/copy.hpp"
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	37
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	38 // Possible sizes for the card counts cache: odd primes that roughly double in size.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	39 // (See jvmtiTagMap.cpp).
2431 02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	40
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	41 #define MAX_SIZE ((size_t) -1)
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	42
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	43 size_t ConcurrentG1Refine::_cc_cache_sizes[] = {
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	44 16381, 32771, 76831, 150001, 307261,
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	45 614563, 1228891, 2457733, 4915219, 9830479,
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	46 19660831, 39321619, 78643219, 157286461, MAX_SIZE
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	47 };
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	48
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	49 ConcurrentG1Refine::ConcurrentG1Refine() :
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	50 _card_counts(NULL), _card_epochs(NULL),
2431 02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	51 _n_card_counts(0), _max_cards(0), _max_n_card_counts(0),
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	52 _cache_size_index(0), _expand_card_counts(false),
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	53 _hot_cache(NULL),
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	54 _def_use_cache(false), _use_cache(false),
2434 c84ee870e0b9 7033292: G1: nightly failure: Non-dirty cards in region that should be dirty tonyp parents: 2431 diff changeset	55 // We initialize the epochs of the array to 0. By initializing
c84ee870e0b9 7033292: G1: nightly failure: Non-dirty cards in region that should be dirty tonyp parents: 2431 diff changeset	56 // _n_periods to 1 and not 0 we automatically invalidate all the
c84ee870e0b9 7033292: G1: nightly failure: Non-dirty cards in region that should be dirty tonyp parents: 2431 diff changeset	57 // entries on the array. Otherwise we might accidentally think that
c84ee870e0b9 7033292: G1: nightly failure: Non-dirty cards in region that should be dirty tonyp parents: 2431 diff changeset	58 // we claimed a card that was in fact never set (see CR7033292).
c84ee870e0b9 7033292: G1: nightly failure: Non-dirty cards in region that should be dirty tonyp parents: 2431 diff changeset	59 _n_periods(1),
794 315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	60 _threads(NULL), _n_threads(0)
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	61 {
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	62
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	63 // Ergomonically select initial concurrent refinement parameters
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	64 if (FLAG_IS_DEFAULT(G1ConcRefinementGreenZone)) {
b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	65 FLAG_SET_DEFAULT(G1ConcRefinementGreenZone, MAX2<int>(ParallelGCThreads, 1));
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	66 }
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	67 set_green_zone(G1ConcRefinementGreenZone);
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	68
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	69 if (FLAG_IS_DEFAULT(G1ConcRefinementYellowZone)) {
b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	70 FLAG_SET_DEFAULT(G1ConcRefinementYellowZone, green_zone() * 3);
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	71 }
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	72 set_yellow_zone(MAX2<int>(G1ConcRefinementYellowZone, green_zone()));
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	73
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	74 if (FLAG_IS_DEFAULT(G1ConcRefinementRedZone)) {
b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	75 FLAG_SET_DEFAULT(G1ConcRefinementRedZone, yellow_zone() * 2);
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	76 }
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	77 set_red_zone(MAX2<int>(G1ConcRefinementRedZone, yellow_zone()));
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	78 _n_worker_threads = thread_num();
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	79 // We need one extra thread to do the young gen rset size sampling.
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	80 _n_threads = _n_worker_threads + 1;
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	81 reset_threshold_step();
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	82
6197 d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 2434 diff changeset	83 _threads = NEW_C_HEAP_ARRAY(ConcurrentG1RefineThread*, _n_threads, mtGC);
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	84 int worker_id_offset = (int)DirtyCardQueueSet::num_par_ids();
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	85 ConcurrentG1RefineThread *next = NULL;
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	86 for (int i = _n_threads - 1; i >= 0; i--) {
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	87 ConcurrentG1RefineThread* t = new ConcurrentG1RefineThread(this, next, worker_id_offset, i);
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	88 assert(t != NULL, "Conc refine should have been created");
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	89 assert(t->cg1r() == this, "Conc refine thread should refer to this");
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	90 _threads[i] = t;
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	91 next = t;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	92 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	93 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	94
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	95 void ConcurrentG1Refine::reset_threshold_step() {
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	96 if (FLAG_IS_DEFAULT(G1ConcRefinementThresholdStep)) {
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	97 _thread_threshold_step = (yellow_zone() - green_zone()) / (worker_thread_num() + 1);
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	98 } else {
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	99 _thread_threshold_step = G1ConcRefinementThresholdStep;
795 215f81b4d9b3 6841831: G1: assert(contains_reference(from),"We just added it!") fires iveresov parents: 794 diff changeset	100 }
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	101 }
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	102
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	103 int ConcurrentG1Refine::thread_num() {
1282 b81f3572f355 6928059: G1: command line parameter renaming tonyp parents: 1246 diff changeset	104 return MAX2<int>((G1ConcRefinementThreads > 0) ? G1ConcRefinementThreads : ParallelGCThreads, 1);
795 215f81b4d9b3 6841831: G1: assert(contains_reference(from),"We just added it!") fires iveresov parents: 794 diff changeset	105 }
215f81b4d9b3 6841831: G1: assert(contains_reference(from),"We just added it!") fires iveresov parents: 794 diff changeset	106
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	107 void ConcurrentG1Refine::init() {
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	108 if (G1ConcRSLogCacheSize > 0) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	109 _g1h = G1CollectedHeap::heap();
2431 02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	110
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	111 _max_cards = _g1h->max_capacity() >> CardTableModRefBS::card_shift;
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	112 _max_n_card_counts = _max_cards * G1MaxHotCardCountSizePercent / 100;
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	113
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	114 size_t max_card_num = ((size_t)1 << (sizeof(unsigned)*BitsPerByte-1)) - 1;
2431 02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	115 guarantee(_max_cards < max_card_num, "card_num representation");
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	116
2431 02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	117 // We need _n_card_counts to be less than _max_n_card_counts here
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	118 // so that the expansion call (below) actually allocates the
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	119 // _counts and _epochs arrays.
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	120 assert(_n_card_counts == 0, "pre-condition");
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	121 assert(_max_n_card_counts > 0, "pre-condition");
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	122
2431 02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	123 // Find the index into cache size array that is of a size that's
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	124 // large enough to hold desired_sz.
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	125 size_t desired_sz = _max_cards / InitialCacheFraction;
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	126 int desired_sz_index = 0;
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	127 while (_cc_cache_sizes[desired_sz_index] < desired_sz) {
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	128 desired_sz_index += 1;
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	129 assert(desired_sz_index < MAX_CC_CACHE_INDEX, "invariant");
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	130 }
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	131 assert(desired_sz_index < MAX_CC_CACHE_INDEX, "invariant");
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	132
2431 02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	133 // If the desired_sz value is between two sizes then
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	134 // _cc_cache_sizes[desired_sz_index-1] < desired_sz <= _cc_cache_sizes[desired_sz_index]
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	135 // we will start with the lower size in the optimistic expectation that
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	136 // we will not need to expand up. Note desired_sz_index could also be 0.
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	137 if (desired_sz_index > 0 &&
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	138 _cc_cache_sizes[desired_sz_index] > desired_sz) {
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	139 desired_sz_index -= 1;
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	140 }
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	141
2431 02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	142 if (!expand_card_count_cache(desired_sz_index)) {
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	143 // Allocation was unsuccessful - exit
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	144 vm_exit_during_initialization("Could not reserve enough space for card count cache");
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	145 }
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	146 assert(_n_card_counts > 0, "post-condition");
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	147 assert(_cache_size_index == desired_sz_index, "post-condition");
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	148
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	149 Copy::fill_to_bytes(&_card_counts[0],
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	150 _n_card_counts * sizeof(CardCountCacheEntry));
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	151 Copy::fill_to_bytes(&_card_epochs[0], _n_card_counts * sizeof(CardEpochCacheEntry));
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	152
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	153 ModRefBarrierSet* bs = _g1h->mr_bs();
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	154 guarantee(bs->is_a(BarrierSet::CardTableModRef), "Precondition");
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	155 _ct_bs = (CardTableModRefBS*)bs;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	156 _ct_bot = _ct_bs->byte_for_const(_g1h->reserved_region().start());
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	157
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	158 _def_use_cache = true;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	159 _use_cache = true;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	160 _hot_cache_size = (1 << G1ConcRSLogCacheSize);
6197 d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 2434 diff changeset	161 _hot_cache = NEW_C_HEAP_ARRAY(jbyte*, _hot_cache_size, mtGC);
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	162 _n_hot = 0;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	163 _hot_cache_idx = 0;
889 15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	164
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	165 // For refining the cards in the hot cache in parallel
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	166 int n_workers = (ParallelGCThreads > 0 ?
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	167 _g1h->workers()->total_workers() : 1);
889 15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	168 _hot_cache_par_chunk_size = MAX2(1, _hot_cache_size / n_workers);
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	169 _hot_cache_par_claimed_idx = 0;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	170 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	171 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	172
794 315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	173 void ConcurrentG1Refine::stop() {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	174 if (_threads != NULL) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	175 for (int i = 0; i < _n_threads; i++) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	176 _threads[i]->stop();
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	177 }
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	178 }
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	179 }
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	180
1111 44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	181 void ConcurrentG1Refine::reinitialize_threads() {
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	182 reset_threshold_step();
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	183 if (_threads != NULL) {
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	184 for (int i = 0; i < _n_threads; i++) {
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	185 _threads[i]->initialize();
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	186 }
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	187 }
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	188 }
44f61c24ddab 6862387: tune concurrent refinement further iveresov parents: 1019 diff changeset	189
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	190 ConcurrentG1Refine::~ConcurrentG1Refine() {
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	191 if (G1ConcRSLogCacheSize > 0) {
2431 02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	192 // Please see the comment in allocate_card_count_cache
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	193 // for why we call os::malloc() and os::free() directly.
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	194 assert(_card_counts != NULL, "Logic");
6197 d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 2434 diff changeset	195 os::free(_card_counts, mtGC);
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	196 assert(_card_epochs != NULL, "Logic");
6197 d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 2434 diff changeset	197 os::free(_card_epochs, mtGC);
2431 02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	198
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	199 assert(_hot_cache != NULL, "Logic");
6197 d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 2434 diff changeset	200 FREE_C_HEAP_ARRAY(jbyte*, _hot_cache, mtGC);
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	201 }
794 315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	202 if (_threads != NULL) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	203 for (int i = 0; i < _n_threads; i++) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	204 delete _threads[i];
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	205 }
6197 d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 2434 diff changeset	206 FREE_C_HEAP_ARRAY(ConcurrentG1RefineThread*, _threads, mtGC);
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	207 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	208 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	209
794 315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	210 void ConcurrentG1Refine::threads_do(ThreadClosure *tc) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	211 if (_threads != NULL) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	212 for (int i = 0; i < _n_threads; i++) {
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	213 tc->do_thread(_threads[i]);
315a5d70b295 6484957: G1: parallel concurrent refinement iveresov parents: 637 diff changeset	214 }
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	215 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	216 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	217
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	218 bool ConcurrentG1Refine::is_young_card(jbyte* card_ptr) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	219 HeapWord* start = _ct_bs->addr_for(card_ptr);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	220 HeapRegion* r = _g1h->heap_region_containing(start);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	221 if (r != NULL && r->is_young()) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	222 return true;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	223 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	224 // This card is not associated with a heap region
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	225 // so can't be young.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	226 return false;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	227 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	228
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	229 jbyte* ConcurrentG1Refine::add_card_count(jbyte* card_ptr, int* count, bool* defer) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	230 unsigned new_card_num = ptr_2_card_num(card_ptr);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	231 unsigned bucket = hash(new_card_num);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	232 assert(0 <= bucket && bucket < _n_card_counts, "Bounds");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	233
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	234 CardCountCacheEntry* count_ptr = &_card_counts[bucket];
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	235 CardEpochCacheEntry* epoch_ptr = &_card_epochs[bucket];
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	236
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	237 // We have to construct a new entry if we haven't updated the counts
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	238 // during the current period, or if the count was updated for a
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	239 // different card number.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	240 unsigned int new_epoch = (unsigned int) _n_periods;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	241 julong new_epoch_entry = make_epoch_entry(new_card_num, new_epoch);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	242
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	243 while (true) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	244 // Fetch the previous epoch value
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	245 julong prev_epoch_entry = epoch_ptr->_value;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	246 julong cas_res;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	247
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	248 if (extract_epoch(prev_epoch_entry) != new_epoch) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	249 // This entry has not yet been updated during this period.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	250 // Note: we update the epoch value atomically to ensure
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	251 // that there is only one winner that updates the cached
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	252 // card_ptr value even though all the refine threads share
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	253 // the same epoch value.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	254
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	255 cas_res = (julong) Atomic::cmpxchg((jlong) new_epoch_entry,
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	256 (volatile jlong*)&epoch_ptr->_value,
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	257 (jlong) prev_epoch_entry);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	258
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	259 if (cas_res == prev_epoch_entry) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	260 // We have successfully won the race to update the
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	261 // epoch and card_num value. Make it look like the
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	262 // count and eviction count were previously cleared.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	263 count_ptr->_count = 1;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	264 count_ptr->_evict_count = 0;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	265 *count = 0;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	266 // We can defer the processing of card_ptr
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	267 *defer = true;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	268 return card_ptr;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	269 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	270 // We did not win the race to update the epoch field, so some other
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	271 // thread must have done it. The value that gets returned by CAS
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	272 // should be the new epoch value.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	273 assert(extract_epoch(cas_res) == new_epoch, "unexpected epoch");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	274 // We could 'continue' here or just re-read the previous epoch value
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	275 prev_epoch_entry = epoch_ptr->_value;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	276 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	277
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	278 // The epoch entry for card_ptr has been updated during this period.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	279 unsigned old_card_num = extract_card_num(prev_epoch_entry);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	280
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	281 // The card count that will be returned to caller
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	282 *count = count_ptr->_count;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	283
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	284 // Are we updating the count for the same card?
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	285 if (new_card_num == old_card_num) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	286 // Same card - just update the count. We could have more than one
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	287 // thread racing to update count for the current card. It should be
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	288 // OK not to use a CAS as the only penalty should be some missed
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	289 // increments of the count which delays identifying the card as "hot".
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	290
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	291 if (*count < max_jubyte) count_ptr->_count++;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	292 // We can defer the processing of card_ptr
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	293 *defer = true;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	294 return card_ptr;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	295 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	296
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	297 // Different card - evict old card info
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	298 if (count_ptr->_evict_count < max_jubyte) count_ptr->_evict_count++;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	299 if (count_ptr->_evict_count > G1CardCountCacheExpandThreshold) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	300 // Trigger a resize the next time we clear
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	301 _expand_card_counts = true;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	302 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	303
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	304 cas_res = (julong) Atomic::cmpxchg((jlong) new_epoch_entry,
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	305 (volatile jlong*)&epoch_ptr->_value,
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	306 (jlong) prev_epoch_entry);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	307
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	308 if (cas_res == prev_epoch_entry) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	309 // We successfully updated the card num value in the epoch entry
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	310 count_ptr->_count = 0; // initialize counter for new card num
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	311 jbyte* old_card_ptr = card_num_2_ptr(old_card_num);
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	312
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	313 // Even though the region containg the card at old_card_num was not
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	314 // in the young list when old_card_num was recorded in the epoch
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	315 // cache it could have been added to the free list and subsequently
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	316 // added to the young list in the intervening time. See CR 6817995.
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	317 // We do not deal with this case here - it will be handled in
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	318 // HeapRegion::oops_on_card_seq_iterate_careful after it has been
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	319 // determined that the region containing the card has been allocated
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	320 // to, and it's safe to check the young type of the region.
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	321
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	322 // We do not want to defer processing of card_ptr in this case
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	323 // (we need to refine old_card_ptr and card_ptr)
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	324 *defer = false;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	325 return old_card_ptr;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	326 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	327 // Someone else beat us - try again.
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	328 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	329 }
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	330
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	331 jbyte* ConcurrentG1Refine::cache_insert(jbyte* card_ptr, bool* defer) {
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	332 int count;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	333 jbyte* cached_ptr = add_card_count(card_ptr, &count, defer);
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	334 assert(cached_ptr != NULL, "bad cached card ptr");
1246 deada8912c54 6914402: G1: assert(!is_young_card(cached_ptr),"shouldn't get a card in young region") johnc parents: 1111 diff changeset	335
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	336 // We've just inserted a card pointer into the card count cache
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	337 // and got back the card that we just inserted or (evicted) the
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	338 // previous contents of that count slot.
1246 deada8912c54 6914402: G1: assert(!is_young_card(cached_ptr),"shouldn't get a card in young region") johnc parents: 1111 diff changeset	339
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	340 // The card we got back could be in a young region. When the
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	341 // returned card (if evicted) was originally inserted, we had
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	342 // determined that its containing region was not young. However
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	343 // it is possible for the region to be freed during a cleanup
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	344 // pause, then reallocated and tagged as young which will result
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	345 // in the returned card residing in a young region.
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	346 //
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	347 // We do not deal with this case here - the change from non-young
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	348 // to young could be observed at any time - it will be handled in
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	349 // HeapRegion::oops_on_card_seq_iterate_careful after it has been
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	350 // determined that the region containing the card has been allocated
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	351 // to.
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	352
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	353 // The card pointer we obtained from card count cache is not hot
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	354 // so do not store it in the cache; return it for immediate
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	355 // refining.
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	356 if (count < G1ConcRSHotCardLimit) {
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	357 return cached_ptr;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	358 }
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	359
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	360 // Otherwise, the pointer we got from the _card_counts cache is hot.
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	361 jbyte* res = NULL;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	362 MutexLockerEx x(HotCardCache_lock, Mutex::_no_safepoint_check_flag);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	363 if (_n_hot == _hot_cache_size) {
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	364 res = _hot_cache[_hot_cache_idx];
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	365 _n_hot--;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	366 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	367 // Now _n_hot < _hot_cache_size, and we can insert at _hot_cache_idx.
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	368 _hot_cache[_hot_cache_idx] = cached_ptr;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	369 _hot_cache_idx++;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	370 if (_hot_cache_idx == _hot_cache_size) _hot_cache_idx = 0;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	371 _n_hot++;
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	372
1666 5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	373 // The card obtained from the hot card cache could be in a young
5cbac8938c4c 6956639: G1: assert(cached_ptr != card_ptr) failed: shouldn't be, concurrentG1Refine.cpp:307 johnc parents: 1552 diff changeset	374 // region. See above on how this can happen.
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	375
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	376 return res;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	377 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	378
1705 2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	379 void ConcurrentG1Refine::clean_up_cache(int worker_i,
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	380 G1RemSet* g1rs,
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	381 DirtyCardQueue* into_cset_dcq) {
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	382 assert(!use_cache(), "cache should be disabled");
889 15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	383 int start_idx;
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	384
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	385 while ((start_idx = _hot_cache_par_claimed_idx) < _n_hot) { // read once
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	386 int end_idx = start_idx + _hot_cache_par_chunk_size;
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	387
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	388 if (start_idx ==
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	389 Atomic::cmpxchg(end_idx, &_hot_cache_par_claimed_idx, start_idx)) {
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	390 // The current worker has successfully claimed the chunk [start_idx..end_idx)
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	391 end_idx = MIN2(end_idx, _n_hot);
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	392 for (int i = start_idx; i < end_idx; i++) {
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	393 jbyte* entry = _hot_cache[i];
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	394 if (entry != NULL) {
1705 2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	395 if (g1rs->concurrentRefineOneCard(entry, worker_i, true)) {
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	396 // 'entry' contains references that point into the current
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	397 // collection set. We need to record 'entry' in the DCQS
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	398 // that's used for that purpose.
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	399 //
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	400 // The only time we care about recording cards that contain
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	401 // references that point into the collection set is during
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	402 // RSet updating while within an evacuation pause.
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	403 // In this case worker_i should be the id of a GC worker thread
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	404 assert(SafepointSynchronize::is_at_safepoint(), "not during an evacuation pause");
2364 04d1138b4cce 7023747: G1: too strict assert in RefineRecordRefsIntoCSCardTableEntryClosure::do_card_ptr in g1RemSet.cpp brutisso parents: 2188 diff changeset	405 assert(worker_i < (int) (ParallelGCThreads == 0 ? 1 : ParallelGCThreads), "incorrect worker id");
1705 2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	406 into_cset_dcq->enqueue(entry);
2d160770d2e5 6814437: G1: remove the _new_refs array johnc parents: 1666 diff changeset	407 }
889 15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	408 }
15c5903cf9e1 6865703: G1: Parallelize hot card cache cleanup johnc parents: 844 diff changeset	409 }
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	410 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	411 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	412 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	413
2431 02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	414 // The arrays used to hold the card counts and the epochs must have
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	415 // a 1:1 correspondence. Hence they are allocated and freed together
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	416 // Returns true if the allocations of both the counts and epochs
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	417 // were successful; false otherwise.
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	418 bool ConcurrentG1Refine::allocate_card_count_cache(size_t n,
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	419 CardCountCacheEntry** counts,
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	420 CardEpochCacheEntry** epochs) {
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	421 // We call the allocation/free routines directly for the counts
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	422 // and epochs arrays. The NEW_C_HEAP_ARRAY/FREE_C_HEAP_ARRAY
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	423 // macros call AllocateHeap and FreeHeap respectively.
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	424 // AllocateHeap will call vm_exit_out_of_memory in the event
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	425 // of an allocation failure and abort the JVM. With the
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	426 // _counts/epochs arrays we only need to abort the JVM if the
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	427 // initial allocation of these arrays fails.
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	428 //
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	429 // Additionally AllocateHeap/FreeHeap do some tracing of
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	430 // allocate/free calls so calling one without calling the
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	431 // other can cause inconsistencies in the tracing. So we
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	432 // call neither.
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	433
2431 02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	434 assert(*counts == NULL, "out param");
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	435 assert(*epochs == NULL, "out param");
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	436
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	437 size_t counts_size = n * sizeof(CardCountCacheEntry);
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	438 size_t epochs_size = n * sizeof(CardEpochCacheEntry);
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	439
6197 d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 2434 diff changeset	440 counts = (CardCountCacheEntry) os::malloc(counts_size, mtGC);
2431 02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	441 if (*counts == NULL) {
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	442 // allocation was unsuccessful
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	443 return false;
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	444 }
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	445
6197 d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 2434 diff changeset	446 epochs = (CardEpochCacheEntry) os::malloc(epochs_size, mtGC);
2431 02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	447 if (*epochs == NULL) {
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	448 // allocation was unsuccessful - free counts array
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	449 assert(*counts != NULL, "must be");
6197 d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 2434 diff changeset	450 os::free(*counts, mtGC);
2431 02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	451 *counts = NULL;
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	452 return false;
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	453 }
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	454
2431 02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	455 // We successfully allocated both counts and epochs
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	456 return true;
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	457 }
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	458
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	459 // Returns true if the card counts/epochs cache was
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	460 // successfully expanded; false otherwise.
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	461 bool ConcurrentG1Refine::expand_card_count_cache(int cache_size_idx) {
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	462 // Can we expand the card count and epoch tables?
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	463 if (_n_card_counts < _max_n_card_counts) {
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	464 assert(cache_size_idx >= 0 && cache_size_idx < MAX_CC_CACHE_INDEX, "oob");
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	465
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	466 size_t cache_size = _cc_cache_sizes[cache_size_idx];
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	467 // Make sure we don't go bigger than we will ever need
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	468 cache_size = MIN2(cache_size, _max_n_card_counts);
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	469
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	470 // Should we expand the card count and card epoch tables?
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	471 if (cache_size > _n_card_counts) {
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	472 // We have been asked to allocate new, larger, arrays for
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	473 // the card counts and the epochs. Attempt the allocation
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	474 // of both before we free the existing arrays in case
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	475 // the allocation is unsuccessful...
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	476 CardCountCacheEntry* counts = NULL;
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	477 CardEpochCacheEntry* epochs = NULL;
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	478
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	479 if (allocate_card_count_cache(cache_size, &counts, &epochs)) {
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	480 // Allocation was successful.
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	481 // We can just free the old arrays; we're
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	482 // not interested in preserving the contents
6197 d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 2434 diff changeset	483 if (_card_counts != NULL) os::free(_card_counts, mtGC);
d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 2434 diff changeset	484 if (_card_epochs != NULL) os::free(_card_epochs, mtGC);
2431 02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	485
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	486 // Cache the size of the arrays and the index that got us there.
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	487 _n_card_counts = cache_size;
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	488 _cache_size_index = cache_size_idx;
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	489
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	490 _card_counts = counts;
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	491 _card_epochs = epochs;
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	492
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	493 // We successfully allocated/expanded the caches.
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	494 return true;
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	495 }
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	496 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	497 }
2431 02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	498
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	499 // We did not successfully expand the caches.
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	500 return false;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	501 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	502
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	503 void ConcurrentG1Refine::clear_and_record_card_counts() {
6219 922993931b3d 7178361: G1: Make sure that PrintGC and PrintGCDetails use the same timing for the GC pause brutisso parents: 2434 diff changeset	504 if (G1ConcRSLogCacheSize == 0) {
922993931b3d 7178361: G1: Make sure that PrintGC and PrintGCDetails use the same timing for the GC pause brutisso parents: 2434 diff changeset	505 return;
922993931b3d 7178361: G1: Make sure that PrintGC and PrintGCDetails use the same timing for the GC pause brutisso parents: 2434 diff changeset	506 }
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	507
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	508 double start = os::elapsedTime();
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	509
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	510 if (_expand_card_counts) {
2431 02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	511 int new_idx = _cache_size_index + 1;
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	512
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	513 if (expand_card_count_cache(new_idx)) {
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	514 // Allocation was successful and _n_card_counts has
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	515 // been updated to the new size. We only need to clear
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	516 // the epochs so we don't read a bogus epoch value
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	517 // when inserting a card into the hot card cache.
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	518 Copy::fill_to_bytes(&_card_epochs[0], _n_card_counts * sizeof(CardEpochCacheEntry));
02f49b66361a 7026932: G1: No need to abort VM when card count cache expansion fails johnc parents: 2364 diff changeset	519 }
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	520 _expand_card_counts = false;
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	521 }
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	522
890 6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	523 int this_epoch = (int) _n_periods;
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	524 assert((this_epoch+1) <= max_jint, "to many periods");
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	525 // Update epoch
6cb8e9df7174 6819077: G1: first GC thread coming late into the GC. johnc parents: 889 diff changeset	526 _n_periods++;
6219 922993931b3d 7178361: G1: Make sure that PrintGC and PrintGCDetails use the same timing for the GC pause brutisso parents: 2434 diff changeset	527 double cc_clear_time_ms = (os::elapsedTime() - start) * 1000;
922993931b3d 7178361: G1: Make sure that PrintGC and PrintGCDetails use the same timing for the GC pause brutisso parents: 2434 diff changeset	528 _g1h->g1_policy()->phase_times()->record_cc_clear_time_ms(cc_clear_time_ms);
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: diff changeset	529 }
1019 035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	530
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	531 void ConcurrentG1Refine::print_worker_threads_on(outputStream* st) const {
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	532 for (int i = 0; i < _n_threads; ++i) {
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	533 _threads[i]->print_on(st);
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	534 st->cr();
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	535 }
035d2e036a9b 6885041: G1: inconsistent thread dump tonyp parents: 890 diff changeset	536 }

Mercurial > hg > truffle

annotate src/share/vm/gc_implementation/g1/concurrentG1Refine.cpp @ 8733:9def4075da6d