truffle: src/share/vm/gc_implementation/g1/concurrentMark.inline.hpp annotate

annotate src/share/vm/gc_implementation/g1/concurrentMark.inline.hpp @ 4837:eff609af17d7

7127706: G1: re-enable survivors during the initial-mark pause Summary: Re-enable survivors during the initial-mark pause. Afterwards, the concurrent marking threads have to scan them and mark everything reachable from them. The next GC will have to wait for the survivors to be scanned. Reviewed-by: brutisso, johnc

author	tonyp
date	Wed, 25 Jan 2012 12:58:23 -0500
parents	d30fa85f9994
children	720b6a76dd9d

rev	line source
3771 842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	1 /*
4787 2ace1c4ee8da 6888336: G1: avoid explicitly marking and pushing objects in survivor spaces tonyp parents: 3776 diff changeset	2 * Copyright (c) 2001, 2012, Oracle and/or its affiliates. All rights reserved.
3771 842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	4 *
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	5 * This code is free software; you can redistribute it and/or modify it
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	6 * under the terms of the GNU General Public License version 2 only, as
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	7 * published by the Free Software Foundation.
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	8 *
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	9 * This code is distributed in the hope that it will be useful, but WITHOUT
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	10 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	11 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	12 * version 2 for more details (a copy is included in the LICENSE file that
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	13 * accompanied this code).
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	14 *
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	15 * You should have received a copy of the GNU General Public License version
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	16 * 2 along with this work; if not, write to the Free Software Foundation,
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	17 * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	18 *
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	19 * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	20 * or visit www.oracle.com if you need additional information or have any
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	21 * questions.
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	22 *
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	23 */
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	24
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	25 #ifndef SHARE_VM_GC_IMPLEMENTATION_G1_CONCURRENTMARK_INLINE_HPP
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	26 #define SHARE_VM_GC_IMPLEMENTATION_G1_CONCURRENTMARK_INLINE_HPP
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	27
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	28 #include "gc_implementation/g1/concurrentMark.hpp"
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	29 #include "gc_implementation/g1/g1CollectedHeap.inline.hpp"
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	30
4836 d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	31 // Returns the index in the liveness accounting card bitmap
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	32 // for the given address
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	33 inline BitMap::idx_t ConcurrentMark::card_bitmap_index_for(HeapWord* addr) {
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	34 // Below, the term "card num" means the result of shifting an address
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	35 // by the card shift -- address 0 corresponds to card number 0. One
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	36 // must subtract the card num of the bottom of the heap to obtain a
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	37 // card table index.
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	38
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	39 intptr_t card_num = intptr_t(uintptr_t(addr) >> CardTableModRefBS::card_shift);
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	40 return card_num - heap_bottom_card_num();
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	41 }
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	42
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	43 // Counts the given memory region in the given task/worker
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	44 // counting data structures.
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	45 inline void ConcurrentMark::count_region(MemRegion mr, HeapRegion* hr,
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	46 size_t* marked_bytes_array,
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	47 BitMap* task_card_bm) {
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	48 G1CollectedHeap* g1h = _g1h;
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	49 HeapWord* start = mr.start();
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	50 HeapWord* last = mr.last();
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	51 size_t region_size_bytes = mr.byte_size();
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	52 size_t index = hr->hrs_index();
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	53
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	54 assert(!hr->continuesHumongous(), "should not be HC region");
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	55 assert(hr == g1h->heap_region_containing(start), "sanity");
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	56 assert(hr == g1h->heap_region_containing(mr.last()), "sanity");
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	57 assert(marked_bytes_array != NULL, "pre-condition");
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	58 assert(task_card_bm != NULL, "pre-condition");
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	59
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	60 // Add to the task local marked bytes for this region.
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	61 marked_bytes_array[index] += region_size_bytes;
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	62
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	63 BitMap::idx_t start_idx = card_bitmap_index_for(start);
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	64 BitMap::idx_t last_idx = card_bitmap_index_for(last);
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	65
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	66 // The card bitmap is task/worker specific => no need to use 'par' routines.
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	67 // Set bits in the inclusive bit range [start_idx, last_idx].
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	68 //
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	69 // For small ranges use a simple loop; otherwise use set_range
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	70 // The range are the cards that are spanned by the object/region
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	71 // so 8 cards will allow objects/regions up to 4K to be handled
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	72 // using the loop.
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	73 if ((last_idx - start_idx) <= 8) {
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	74 for (BitMap::idx_t i = start_idx; i <= last_idx; i += 1) {
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	75 task_card_bm->set_bit(i);
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	76 }
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	77 } else {
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	78 assert(last_idx < task_card_bm->size(), "sanity");
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	79 // Note: BitMap::set_range() is exclusive.
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	80 task_card_bm->set_range(start_idx, last_idx+1);
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	81 }
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	82 }
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	83
4837 eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	84 // Counts the given memory region in the task/worker counting
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	85 // data structures for the given worker id.
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	86 inline void ConcurrentMark::count_region(MemRegion mr,
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	87 HeapRegion* hr,
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	88 uint worker_id) {
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	89 size_t* marked_bytes_array = count_marked_bytes_array_for(worker_id);
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	90 BitMap* task_card_bm = count_card_bitmap_for(worker_id);
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	91 count_region(mr, hr, marked_bytes_array, task_card_bm);
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	92 }
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	93
4836 d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	94 // Counts the given memory region, which may be a single object, in the
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	95 // task/worker counting data structures for the given worker id.
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	96 inline void ConcurrentMark::count_region(MemRegion mr, uint worker_id) {
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	97 HeapWord* addr = mr.start();
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	98 HeapRegion* hr = _g1h->heap_region_containing_raw(addr);
4837 eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	99 count_region(mr, hr, worker_id);
4836 d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	100 }
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	101
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	102 // Counts the given object in the given task/worker counting data structures.
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	103 inline void ConcurrentMark::count_object(oop obj,
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	104 HeapRegion* hr,
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	105 size_t* marked_bytes_array,
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	106 BitMap* task_card_bm) {
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	107 MemRegion mr((HeapWord*)obj, obj->size());
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	108 count_region(mr, hr, marked_bytes_array, task_card_bm);
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	109 }
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	110
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	111 // Counts the given object in the task/worker counting data
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	112 // structures for the given worker id.
4837 eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	113 inline void ConcurrentMark::count_object(oop obj,
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	114 HeapRegion* hr,
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	115 uint worker_id) {
4836 d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	116 size_t* marked_bytes_array = count_marked_bytes_array_for(worker_id);
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	117 BitMap* task_card_bm = count_card_bitmap_for(worker_id);
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	118 HeapWord* addr = (HeapWord*) obj;
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	119 count_object(obj, hr, marked_bytes_array, task_card_bm);
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	120 }
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	121
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	122 // Attempts to mark the given object and, if successful, counts
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	123 // the object in the given task/worker counting structures.
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	124 inline bool ConcurrentMark::par_mark_and_count(oop obj,
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	125 HeapRegion* hr,
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	126 size_t* marked_bytes_array,
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	127 BitMap* task_card_bm) {
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	128 HeapWord* addr = (HeapWord*)obj;
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	129 if (_nextMarkBitMap->parMark(addr)) {
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	130 // Update the task specific count data for the object.
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	131 count_object(obj, hr, marked_bytes_array, task_card_bm);
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	132 return true;
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	133 }
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	134 return false;
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	135 }
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	136
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	137 // Attempts to mark the given object and, if successful, counts
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	138 // the object in the task/worker counting structures for the
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	139 // given worker id.
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	140 inline bool ConcurrentMark::par_mark_and_count(oop obj,
4837 eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	141 size_t word_size,
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	142 HeapRegion* hr,
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	143 uint worker_id) {
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	144 HeapWord* addr = (HeapWord*)obj;
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	145 if (_nextMarkBitMap->parMark(addr)) {
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	146 MemRegion mr(addr, word_size);
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	147 count_region(mr, hr, worker_id);
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	148 return true;
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	149 }
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	150 return false;
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	151 }
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	152
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	153 // Attempts to mark the given object and, if successful, counts
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	154 // the object in the task/worker counting structures for the
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	155 // given worker id.
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	156 inline bool ConcurrentMark::par_mark_and_count(oop obj,
4836 d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	157 HeapRegion* hr,
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	158 uint worker_id) {
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	159 HeapWord* addr = (HeapWord*)obj;
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	160 if (_nextMarkBitMap->parMark(addr)) {
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	161 // Update the task specific count data for the object.
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	162 count_object(obj, hr, worker_id);
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	163 return true;
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	164 }
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	165 return false;
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	166 }
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	167
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	168 // As above - but we don't know the heap region containing the
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	169 // object and so have to supply it.
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	170 inline bool ConcurrentMark::par_mark_and_count(oop obj, uint worker_id) {
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	171 HeapWord* addr = (HeapWord*)obj;
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	172 HeapRegion* hr = _g1h->heap_region_containing_raw(addr);
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	173 return par_mark_and_count(obj, hr, worker_id);
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	174 }
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	175
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	176 // Similar to the above routine but we already know the size, in words, of
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	177 // the object that we wish to mark/count
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	178 inline bool ConcurrentMark::par_mark_and_count(oop obj,
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	179 size_t word_size,
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	180 uint worker_id) {
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	181 HeapWord* addr = (HeapWord*)obj;
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	182 if (_nextMarkBitMap->parMark(addr)) {
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	183 // Update the task specific count data for the object.
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	184 MemRegion mr(addr, word_size);
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	185 count_region(mr, worker_id);
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	186 return true;
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	187 }
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	188 return false;
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	189 }
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	190
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	191 // Unconditionally mark the given object, and unconditinally count
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	192 // the object in the counting structures for worker id 0.
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	193 // Should not be called from parallel code.
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	194 inline bool ConcurrentMark::mark_and_count(oop obj, HeapRegion* hr) {
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	195 HeapWord* addr = (HeapWord*)obj;
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	196 _nextMarkBitMap->mark(addr);
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	197 // Update the task specific count data for the object.
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	198 count_object(obj, hr, 0 /* worker_id */);
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	199 return true;
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	200 }
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	201
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	202 // As above - but we don't have the heap region containing the
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	203 // object, so we have to supply it.
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	204 inline bool ConcurrentMark::mark_and_count(oop obj) {
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	205 HeapWord* addr = (HeapWord*)obj;
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	206 HeapRegion* hr = _g1h->heap_region_containing_raw(addr);
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	207 return mark_and_count(obj, hr);
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	208 }
d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	209
4827 2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	210 inline bool CMBitMapRO::iterate(BitMapClosure* cl, MemRegion mr) {
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	211 HeapWord* start_addr = MAX2(startWord(), mr.start());
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	212 HeapWord* end_addr = MIN2(endWord(), mr.end());
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	213
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	214 if (end_addr > start_addr) {
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	215 // Right-open interval [start-offset, end-offset).
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	216 BitMap::idx_t start_offset = heapWordToOffset(start_addr);
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	217 BitMap::idx_t end_offset = heapWordToOffset(end_addr);
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	218
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	219 start_offset = _bm.get_next_one_offset(start_offset, end_offset);
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	220 while (start_offset < end_offset) {
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	221 HeapWord* obj_addr = offsetToHeapWord(start_offset);
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	222 oop obj = (oop) obj_addr;
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	223 if (!cl->do_bit(start_offset)) {
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	224 return false;
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	225 }
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	226 HeapWord* next_addr = MIN2(obj_addr + obj->size(), end_addr);
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	227 BitMap::idx_t next_offset = heapWordToOffset(next_addr);
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	228 start_offset = _bm.get_next_one_offset(next_offset, end_offset);
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	229 }
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	230 }
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	231 return true;
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	232 }
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	233
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	234 inline bool CMBitMapRO::iterate(BitMapClosure* cl) {
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	235 MemRegion mr(startWord(), sizeInWords());
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	236 return iterate(cl, mr);
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	237 }
2e966d967c5c 7121547: G1: High number mispredicted branches while iterating over the marking bitmap johnc parents: 4787 diff changeset	238
3771 842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	239 inline void CMTask::push(oop obj) {
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	240 HeapWord* objAddr = (HeapWord*) obj;
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	241 assert(_g1h->is_in_g1_reserved(objAddr), "invariant");
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	242 assert(!_g1h->is_on_master_free_list(
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	243 _g1h->heap_region_containing((HeapWord*) objAddr)), "invariant");
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	244 assert(!_g1h->is_obj_ill(obj), "invariant");
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	245 assert(_nextMarkBitMap->isMarked(objAddr), "invariant");
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	246
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	247 if (_cm->verbose_high()) {
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	248 gclog_or_tty->print_cr("[%d] pushing "PTR_FORMAT, _task_id, (void*) obj);
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	249 }
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	250
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	251 if (!_task_queue->push(obj)) {
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	252 // The local task queue looks full. We need to push some entries
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	253 // to the global stack.
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	254
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	255 if (_cm->verbose_medium()) {
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	256 gclog_or_tty->print_cr("[%d] task queue overflow, "
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	257 "moving entries to the global stack",
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	258 _task_id);
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	259 }
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	260 move_entries_to_global_stack();
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	261
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	262 // this should succeed since, even if we overflow the global
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	263 // stack, we should have definitely removed some entries from the
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	264 // local queue. So, there must be space on it.
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	265 bool success = _task_queue->push(obj);
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	266 assert(success, "invariant");
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	267 }
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	268
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	269 statsOnly( int tmp_size = _task_queue->size();
3776 23d434c6290d 7055073: G1: code cleanup in the concurrentMark.* files tonyp parents: 3771 diff changeset	270 if (tmp_size > _local_max_size) {
3771 842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	271 _local_max_size = tmp_size;
3776 23d434c6290d 7055073: G1: code cleanup in the concurrentMark.* files tonyp parents: 3771 diff changeset	272 }
3771 842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	273 ++_local_pushes );
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	274 }
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	275
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	276 // This determines whether the method below will check both the local
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	277 // and global fingers when determining whether to push on the stack a
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	278 // gray object (value 1) or whether it will only check the global one
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	279 // (value 0). The tradeoffs are that the former will be a bit more
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	280 // accurate and possibly push less on the stack, but it might also be
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	281 // a little bit slower.
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	282
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	283 #define _CHECK_BOTH_FINGERS_ 1
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	284
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	285 inline void CMTask::deal_with_reference(oop obj) {
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	286 if (_cm->verbose_high()) {
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	287 gclog_or_tty->print_cr("[%d] we're dealing with reference = "PTR_FORMAT,
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	288 _task_id, (void*) obj);
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	289 }
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	290
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	291 ++_refs_reached;
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	292
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	293 HeapWord* objAddr = (HeapWord*) obj;
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	294 assert(obj->is_oop_or_null(true /* ignore mark word */), "Error");
4836 d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	295 if (_g1h->is_in_g1_reserved(objAddr)) {
3771 842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	296 assert(obj != NULL, "null check is implicit");
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	297 if (!_nextMarkBitMap->isMarked(objAddr)) {
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	298 // Only get the containing region if the object is not marked on the
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	299 // bitmap (otherwise, it's a waste of time since we won't do
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	300 // anything with it).
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	301 HeapRegion* hr = _g1h->heap_region_containing_raw(obj);
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	302 if (!hr->obj_allocated_since_next_marking(obj)) {
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	303 if (_cm->verbose_high()) {
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	304 gclog_or_tty->print_cr("[%d] "PTR_FORMAT" is not considered marked",
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	305 _task_id, (void*) obj);
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	306 }
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	307
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	308 // we need to mark it first
4836 d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	309 if (_cm->par_mark_and_count(obj, hr, _marked_bytes_array, _card_bm)) {
3771 842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	310 // No OrderAccess:store_load() is needed. It is implicit in the
4836 d30fa85f9994 6484965: G1: piggy-back liveness accounting phase on marking johnc parents: 4827 diff changeset	311 // CAS done in CMBitMap::parMark() call in the routine above.
3771 842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	312 HeapWord* global_finger = _cm->finger();
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	313
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	314 #if _CHECK_BOTH_FINGERS_
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	315 // we will check both the local and global fingers
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	316
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	317 if (_finger != NULL && objAddr < _finger) {
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	318 if (_cm->verbose_high()) {
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	319 gclog_or_tty->print_cr("[%d] below the local finger ("PTR_FORMAT"), "
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	320 "pushing it", _task_id, _finger);
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	321 }
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	322 push(obj);
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	323 } else if (_curr_region != NULL && objAddr < _region_limit) {
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	324 // do nothing
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	325 } else if (objAddr < global_finger) {
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	326 // Notice that the global finger might be moving forward
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	327 // concurrently. This is not a problem. In the worst case, we
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	328 // mark the object while it is above the global finger and, by
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	329 // the time we read the global finger, it has moved forward
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	330 // passed this object. In this case, the object will probably
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	331 // be visited when a task is scanning the region and will also
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	332 // be pushed on the stack. So, some duplicate work, but no
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	333 // correctness problems.
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	334
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	335 if (_cm->verbose_high()) {
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	336 gclog_or_tty->print_cr("[%d] below the global finger "
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	337 "("PTR_FORMAT"), pushing it",
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	338 _task_id, global_finger);
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	339 }
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	340 push(obj);
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	341 } else {
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	342 // do nothing
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	343 }
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	344 #else // _CHECK_BOTH_FINGERS_
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	345 // we will only check the global finger
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	346
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	347 if (objAddr < global_finger) {
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	348 // see long comment above
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	349
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	350 if (_cm->verbose_high()) {
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	351 gclog_or_tty->print_cr("[%d] below the global finger "
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	352 "("PTR_FORMAT"), pushing it",
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	353 _task_id, global_finger);
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	354 }
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	355 push(obj);
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	356 }
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	357 #endif // _CHECK_BOTH_FINGERS_
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	358 }
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	359 }
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	360 }
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	361 }
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	362 }
842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	363
4787 2ace1c4ee8da 6888336: G1: avoid explicitly marking and pushing objects in survivor spaces tonyp parents: 3776 diff changeset	364 inline void ConcurrentMark::markPrev(oop p) {
2ace1c4ee8da 6888336: G1: avoid explicitly marking and pushing objects in survivor spaces tonyp parents: 3776 diff changeset	365 assert(!_prevMarkBitMap->isMarked((HeapWord*) p), "sanity");
2ace1c4ee8da 6888336: G1: avoid explicitly marking and pushing objects in survivor spaces tonyp parents: 3776 diff changeset	366 // Note we are overriding the read-only view of the prev map here, via
2ace1c4ee8da 6888336: G1: avoid explicitly marking and pushing objects in survivor spaces tonyp parents: 3776 diff changeset	367 // the cast.
2ace1c4ee8da 6888336: G1: avoid explicitly marking and pushing objects in survivor spaces tonyp parents: 3776 diff changeset	368 ((CMBitMap)_prevMarkBitMap)->mark((HeapWord) p);
2ace1c4ee8da 6888336: G1: avoid explicitly marking and pushing objects in survivor spaces tonyp parents: 3776 diff changeset	369 }
2ace1c4ee8da 6888336: G1: avoid explicitly marking and pushing objects in survivor spaces tonyp parents: 3776 diff changeset	370
4837 eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	371 inline void ConcurrentMark::grayRoot(oop obj, size_t word_size,
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	372 uint worker_id, HeapRegion* hr) {
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	373 assert(obj != NULL, "pre-condition");
4787 2ace1c4ee8da 6888336: G1: avoid explicitly marking and pushing objects in survivor spaces tonyp parents: 3776 diff changeset	374 HeapWord* addr = (HeapWord*) obj;
4837 eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	375 if (hr == NULL) {
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	376 hr = _g1h->heap_region_containing_raw(addr);
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	377 } else {
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	378 assert(hr->is_in(addr), "pre-condition");
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	379 }
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	380 assert(hr != NULL, "sanity");
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	381 // Given that we're looking for a region that contains an object
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	382 // header it's impossible to get back a HC region.
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	383 assert(!hr->continuesHumongous(), "sanity");
4787 2ace1c4ee8da 6888336: G1: avoid explicitly marking and pushing objects in survivor spaces tonyp parents: 3776 diff changeset	384
2ace1c4ee8da 6888336: G1: avoid explicitly marking and pushing objects in survivor spaces tonyp parents: 3776 diff changeset	385 // We cannot assert that word_size == obj->size() given that obj
2ace1c4ee8da 6888336: G1: avoid explicitly marking and pushing objects in survivor spaces tonyp parents: 3776 diff changeset	386 // might not be in a consistent state (another thread might be in
2ace1c4ee8da 6888336: G1: avoid explicitly marking and pushing objects in survivor spaces tonyp parents: 3776 diff changeset	387 // the process of copying it). So the best thing we can do is to
2ace1c4ee8da 6888336: G1: avoid explicitly marking and pushing objects in survivor spaces tonyp parents: 3776 diff changeset	388 // assert that word_size is under an upper bound which is its
2ace1c4ee8da 6888336: G1: avoid explicitly marking and pushing objects in survivor spaces tonyp parents: 3776 diff changeset	389 // containing region's capacity.
2ace1c4ee8da 6888336: G1: avoid explicitly marking and pushing objects in survivor spaces tonyp parents: 3776 diff changeset	390 assert(word_size * HeapWordSize <= hr->capacity(),
2ace1c4ee8da 6888336: G1: avoid explicitly marking and pushing objects in survivor spaces tonyp parents: 3776 diff changeset	391 err_msg("size: "SIZE_FORMAT" capacity: "SIZE_FORMAT" "HR_FORMAT,
2ace1c4ee8da 6888336: G1: avoid explicitly marking and pushing objects in survivor spaces tonyp parents: 3776 diff changeset	392 word_size * HeapWordSize, hr->capacity(),
2ace1c4ee8da 6888336: G1: avoid explicitly marking and pushing objects in survivor spaces tonyp parents: 3776 diff changeset	393 HR_FORMAT_PARAMS(hr)));
2ace1c4ee8da 6888336: G1: avoid explicitly marking and pushing objects in survivor spaces tonyp parents: 3776 diff changeset	394
4837 eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	395 if (addr < hr->next_top_at_mark_start()) {
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	396 if (!_nextMarkBitMap->isMarked(addr)) {
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	397 par_mark_and_count(obj, word_size, hr, worker_id);
eff609af17d7 7127706: G1: re-enable survivors during the initial-mark pause tonyp parents: 4836 diff changeset	398 }
4787 2ace1c4ee8da 6888336: G1: avoid explicitly marking and pushing objects in survivor spaces tonyp parents: 3776 diff changeset	399 }
2ace1c4ee8da 6888336: G1: avoid explicitly marking and pushing objects in survivor spaces tonyp parents: 3776 diff changeset	400 }
2ace1c4ee8da 6888336: G1: avoid explicitly marking and pushing objects in survivor spaces tonyp parents: 3776 diff changeset	401
3771 842b840e67db 7046558: G1: concurrent marking optimizations tonyp parents: diff changeset	402 #endif // SHARE_VM_GC_IMPLEMENTATION_G1_CONCURRENTMARK_INLINE_HPP

Mercurial > hg > truffle

annotate src/share/vm/gc_implementation/g1/concurrentMark.inline.hpp @ 4837:eff609af17d7