truffle: src/share/vm/opto/macro.cpp annotate

annotate src/share/vm/opto/macro.cpp @ 14649:f6301b007a16

6498581: ThreadInterruptTest3 produces wrong output on Windows Summary: There is race condition between os::interrupt and os::is_interrupted on Windows. In JVM_Sleep(Thread.sleep), check if thread gets interrupted, it may see interrupted but not really interrupted so cause spurious waking up (early return from sleep). Fix by checking if interrupt event really gets set thus prevent false return. For intrinsic of _isInterrupted, on Windows, go fastpath only on bit not set. Reviewed-by: acorn, kvn Contributed-by: david.holmes@oracle.com, yumin.qi@oracle.com

author	minqi
date	Wed, 26 Feb 2014 15:20:41 -0800
parents	abec000618bf
children	4ca6dc0799b6 62c54fcc0a35

rev	line source
0 a61af66fc99e Initial load duke parents: diff changeset	1 /*
14223 de6a9e811145 8029233: Update copyright year to match last edit in jdk8 hotspot repository for 2013 mikael parents: 12226 diff changeset	2 * Copyright (c) 2005, 2013, Oracle and/or its affiliates. All rights reserved.
0 a61af66fc99e Initial load duke parents: diff changeset	3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
a61af66fc99e Initial load duke parents: diff changeset	4 *
a61af66fc99e Initial load duke parents: diff changeset	5 * This code is free software; you can redistribute it and/or modify it
a61af66fc99e Initial load duke parents: diff changeset	6 * under the terms of the GNU General Public License version 2 only, as
a61af66fc99e Initial load duke parents: diff changeset	7 * published by the Free Software Foundation.
a61af66fc99e Initial load duke parents: diff changeset	8 *
a61af66fc99e Initial load duke parents: diff changeset	9 * This code is distributed in the hope that it will be useful, but WITHOUT
a61af66fc99e Initial load duke parents: diff changeset	10 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
a61af66fc99e Initial load duke parents: diff changeset	11 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
a61af66fc99e Initial load duke parents: diff changeset	12 * version 2 for more details (a copy is included in the LICENSE file that
a61af66fc99e Initial load duke parents: diff changeset	13 * accompanied this code).
a61af66fc99e Initial load duke parents: diff changeset	14 *
a61af66fc99e Initial load duke parents: diff changeset	15 * You should have received a copy of the GNU General Public License version
a61af66fc99e Initial load duke parents: diff changeset	16 * 2 along with this work; if not, write to the Free Software Foundation,
a61af66fc99e Initial load duke parents: diff changeset	17 * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
a61af66fc99e Initial load duke parents: diff changeset	18 *
1552 c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1367 diff changeset	19 * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1367 diff changeset	20 * or visit www.oracle.com if you need additional information or have any
c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1367 diff changeset	21 * questions.
0 a61af66fc99e Initial load duke parents: diff changeset	22 *
a61af66fc99e Initial load duke parents: diff changeset	23 */
a61af66fc99e Initial load duke parents: diff changeset	24
1972 f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	25 #include "precompiled.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	26 #include "compiler/compileLog.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	27 #include "libadt/vectset.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	28 #include "opto/addnode.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	29 #include "opto/callnode.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	30 #include "opto/cfgnode.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	31 #include "opto/compile.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	32 #include "opto/connode.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	33 #include "opto/locknode.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	34 #include "opto/loopnode.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	35 #include "opto/macro.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	36 #include "opto/memnode.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	37 #include "opto/node.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	38 #include "opto/phaseX.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	39 #include "opto/rootnode.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	40 #include "opto/runtime.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	41 #include "opto/subnode.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	42 #include "opto/type.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	43 #include "runtime/sharedRuntime.hpp"
0 a61af66fc99e Initial load duke parents: diff changeset	44
a61af66fc99e Initial load duke parents: diff changeset	45
a61af66fc99e Initial load duke parents: diff changeset	46 //
a61af66fc99e Initial load duke parents: diff changeset	47 // Replace any references to "oldref" in inputs to "use" with "newref".
a61af66fc99e Initial load duke parents: diff changeset	48 // Returns the number of replacements made.
a61af66fc99e Initial load duke parents: diff changeset	49 //
a61af66fc99e Initial load duke parents: diff changeset	50 int PhaseMacroExpand::replace_input(Node use, Node oldref, Node *newref) {
a61af66fc99e Initial load duke parents: diff changeset	51 int nreplacements = 0;
a61af66fc99e Initial load duke parents: diff changeset	52 uint req = use->req();
a61af66fc99e Initial load duke parents: diff changeset	53 for (uint j = 0; j < use->len(); j++) {
a61af66fc99e Initial load duke parents: diff changeset	54 Node *uin = use->in(j);
a61af66fc99e Initial load duke parents: diff changeset	55 if (uin == oldref) {
a61af66fc99e Initial load duke parents: diff changeset	56 if (j < req)
a61af66fc99e Initial load duke parents: diff changeset	57 use->set_req(j, newref);
a61af66fc99e Initial load duke parents: diff changeset	58 else
a61af66fc99e Initial load duke parents: diff changeset	59 use->set_prec(j, newref);
a61af66fc99e Initial load duke parents: diff changeset	60 nreplacements++;
a61af66fc99e Initial load duke parents: diff changeset	61 } else if (j >= req && uin == NULL) {
a61af66fc99e Initial load duke parents: diff changeset	62 break;
a61af66fc99e Initial load duke parents: diff changeset	63 }
a61af66fc99e Initial load duke parents: diff changeset	64 }
a61af66fc99e Initial load duke parents: diff changeset	65 return nreplacements;
a61af66fc99e Initial load duke parents: diff changeset	66 }
a61af66fc99e Initial load duke parents: diff changeset	67
a61af66fc99e Initial load duke parents: diff changeset	68 void PhaseMacroExpand::copy_call_debug_info(CallNode oldcall, CallNode newcall) {
a61af66fc99e Initial load duke parents: diff changeset	69 // Copy debug information and adjust JVMState information
a61af66fc99e Initial load duke parents: diff changeset	70 uint old_dbg_start = oldcall->tf()->domain()->cnt();
a61af66fc99e Initial load duke parents: diff changeset	71 uint new_dbg_start = newcall->tf()->domain()->cnt();
a61af66fc99e Initial load duke parents: diff changeset	72 int jvms_adj = new_dbg_start - old_dbg_start;
a61af66fc99e Initial load duke parents: diff changeset	73 assert (new_dbg_start == newcall->req(), "argument count mismatch");
63 eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	74
12158 766fac3395d6 8012972: Incremental Inlining should support scalar replaced object in debug info kvn parents: 10279 diff changeset	75 // SafePointScalarObject node could be referenced several times in debug info.
766fac3395d6 8012972: Incremental Inlining should support scalar replaced object in debug info kvn parents: 10279 diff changeset	76 // Use Dict to record cloned nodes.
63 eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	77 Dict* sosn_map = new Dict(cmpkey,hashkey);
0 a61af66fc99e Initial load duke parents: diff changeset	78 for (uint i = old_dbg_start; i < oldcall->req(); i++) {
63 eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	79 Node* old_in = oldcall->in(i);
eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	80 // Clone old SafePointScalarObjectNodes, adjusting their field contents.
460 424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	81 if (old_in != NULL && old_in->is_SafePointScalarObject()) {
63 eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	82 SafePointScalarObjectNode* old_sosn = old_in->as_SafePointScalarObject();
eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	83 uint old_unique = C->unique();
12158 766fac3395d6 8012972: Incremental Inlining should support scalar replaced object in debug info kvn parents: 10279 diff changeset	84 Node* new_in = old_sosn->clone(sosn_map);
766fac3395d6 8012972: Incremental Inlining should support scalar replaced object in debug info kvn parents: 10279 diff changeset	85 if (old_unique != C->unique()) { // New node?
4115 1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	86 new_in->set_req(0, C->root()); // reset control edge
63 eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	87 new_in = transform_later(new_in); // Register new node.
eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	88 }
eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	89 old_in = new_in;
eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	90 }
eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	91 newcall->add_req(old_in);
0 a61af66fc99e Initial load duke parents: diff changeset	92 }
63 eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	93
0 a61af66fc99e Initial load duke parents: diff changeset	94 newcall->set_jvms(oldcall->jvms());
a61af66fc99e Initial load duke parents: diff changeset	95 for (JVMState *jvms = newcall->jvms(); jvms != NULL; jvms = jvms->caller()) {
a61af66fc99e Initial load duke parents: diff changeset	96 jvms->set_map(newcall);
a61af66fc99e Initial load duke parents: diff changeset	97 jvms->set_locoff(jvms->locoff()+jvms_adj);
a61af66fc99e Initial load duke parents: diff changeset	98 jvms->set_stkoff(jvms->stkoff()+jvms_adj);
a61af66fc99e Initial load duke parents: diff changeset	99 jvms->set_monoff(jvms->monoff()+jvms_adj);
63 eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	100 jvms->set_scloff(jvms->scloff()+jvms_adj);
0 a61af66fc99e Initial load duke parents: diff changeset	101 jvms->set_endoff(jvms->endoff()+jvms_adj);
a61af66fc99e Initial load duke parents: diff changeset	102 }
a61af66fc99e Initial load duke parents: diff changeset	103 }
a61af66fc99e Initial load duke parents: diff changeset	104
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	105 Node* PhaseMacroExpand::opt_bits_test(Node* ctrl, Node* region, int edge, Node* word, int mask, int bits, bool return_fast_path) {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	106 Node* cmp;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	107 if (mask != 0) {
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	108 Node* and_node = transform_later(new (C) AndXNode(word, MakeConX(mask)));
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	109 cmp = transform_later(new (C) CmpXNode(and_node, MakeConX(bits)));
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	110 } else {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	111 cmp = word;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	112 }
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	113 Node* bol = transform_later(new (C) BoolNode(cmp, BoolTest::ne));
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	114 IfNode* iff = new (C) IfNode( ctrl, bol, PROB_MIN, COUNT_UNKNOWN );
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	115 transform_later(iff);
0 a61af66fc99e Initial load duke parents: diff changeset	116
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	117 // Fast path taken.
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	118 Node *fast_taken = transform_later( new (C) IfFalseNode(iff) );
0 a61af66fc99e Initial load duke parents: diff changeset	119
a61af66fc99e Initial load duke parents: diff changeset	120 // Fast path not-taken, i.e. slow path
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	121 Node *slow_taken = transform_later( new (C) IfTrueNode(iff) );
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	122
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	123 if (return_fast_path) {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	124 region->init_req(edge, slow_taken); // Capture slow-control
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	125 return fast_taken;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	126 } else {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	127 region->init_req(edge, fast_taken); // Capture fast-control
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	128 return slow_taken;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	129 }
0 a61af66fc99e Initial load duke parents: diff changeset	130 }
a61af66fc99e Initial load duke parents: diff changeset	131
a61af66fc99e Initial load duke parents: diff changeset	132 //--------------------copy_predefined_input_for_runtime_call--------------------
a61af66fc99e Initial load duke parents: diff changeset	133 void PhaseMacroExpand::copy_predefined_input_for_runtime_call(Node * ctrl, CallNode* oldcall, CallNode* call) {
a61af66fc99e Initial load duke parents: diff changeset	134 // Set fixed predefined input arguments
a61af66fc99e Initial load duke parents: diff changeset	135 call->init_req( TypeFunc::Control, ctrl );
a61af66fc99e Initial load duke parents: diff changeset	136 call->init_req( TypeFunc::I_O , oldcall->in( TypeFunc::I_O) );
a61af66fc99e Initial load duke parents: diff changeset	137 call->init_req( TypeFunc::Memory , oldcall->in( TypeFunc::Memory ) ); // ?????
a61af66fc99e Initial load duke parents: diff changeset	138 call->init_req( TypeFunc::ReturnAdr, oldcall->in( TypeFunc::ReturnAdr ) );
a61af66fc99e Initial load duke parents: diff changeset	139 call->init_req( TypeFunc::FramePtr, oldcall->in( TypeFunc::FramePtr ) );
a61af66fc99e Initial load duke parents: diff changeset	140 }
a61af66fc99e Initial load duke parents: diff changeset	141
a61af66fc99e Initial load duke parents: diff changeset	142 //------------------------------make_slow_call---------------------------------
a61af66fc99e Initial load duke parents: diff changeset	143 CallNode* PhaseMacroExpand::make_slow_call(CallNode oldcall, const TypeFunc slow_call_type, address slow_call, const char* leaf_name, Node* slow_path, Node* parm0, Node* parm1) {
a61af66fc99e Initial load duke parents: diff changeset	144
a61af66fc99e Initial load duke parents: diff changeset	145 // Slow-path call
a61af66fc99e Initial load duke parents: diff changeset	146 CallNode *call = leaf_name
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	147 ? (CallNode*)new (C) CallLeafNode ( slow_call_type, slow_call, leaf_name, TypeRawPtr::BOTTOM )
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	148 : (CallNode*)new (C) CallStaticJavaNode( slow_call_type, slow_call, OptoRuntime::stub_name(slow_call), oldcall->jvms()->bci(), TypeRawPtr::BOTTOM );
0 a61af66fc99e Initial load duke parents: diff changeset	149
a61af66fc99e Initial load duke parents: diff changeset	150 // Slow path call has no side-effects, uses few values
a61af66fc99e Initial load duke parents: diff changeset	151 copy_predefined_input_for_runtime_call(slow_path, oldcall, call );
a61af66fc99e Initial load duke parents: diff changeset	152 if (parm0 != NULL) call->init_req(TypeFunc::Parms+0, parm0);
a61af66fc99e Initial load duke parents: diff changeset	153 if (parm1 != NULL) call->init_req(TypeFunc::Parms+1, parm1);
a61af66fc99e Initial load duke parents: diff changeset	154 copy_call_debug_info(oldcall, call);
a61af66fc99e Initial load duke parents: diff changeset	155 call->set_cnt(PROB_UNLIKELY_MAG(4)); // Same effect as RC_UNCOMMON.
1621 6027dddc26c6 6677629: PhaseIterGVN::subsume_node() should call hash_delete() and add_users_to_worklist() kvn parents: 1609 diff changeset	156 _igvn.replace_node(oldcall, call);
0 a61af66fc99e Initial load duke parents: diff changeset	157 transform_later(call);
a61af66fc99e Initial load duke parents: diff changeset	158
a61af66fc99e Initial load duke parents: diff changeset	159 return call;
a61af66fc99e Initial load duke parents: diff changeset	160 }
a61af66fc99e Initial load duke parents: diff changeset	161
a61af66fc99e Initial load duke parents: diff changeset	162 void PhaseMacroExpand::extract_call_projections(CallNode *call) {
a61af66fc99e Initial load duke parents: diff changeset	163 _fallthroughproj = NULL;
a61af66fc99e Initial load duke parents: diff changeset	164 _fallthroughcatchproj = NULL;
a61af66fc99e Initial load duke parents: diff changeset	165 _ioproj_fallthrough = NULL;
a61af66fc99e Initial load duke parents: diff changeset	166 _ioproj_catchall = NULL;
a61af66fc99e Initial load duke parents: diff changeset	167 _catchallcatchproj = NULL;
a61af66fc99e Initial load duke parents: diff changeset	168 _memproj_fallthrough = NULL;
a61af66fc99e Initial load duke parents: diff changeset	169 _memproj_catchall = NULL;
a61af66fc99e Initial load duke parents: diff changeset	170 _resproj = NULL;
a61af66fc99e Initial load duke parents: diff changeset	171 for (DUIterator_Fast imax, i = call->fast_outs(imax); i < imax; i++) {
a61af66fc99e Initial load duke parents: diff changeset	172 ProjNode *pn = call->fast_out(i)->as_Proj();
a61af66fc99e Initial load duke parents: diff changeset	173 switch (pn->_con) {
a61af66fc99e Initial load duke parents: diff changeset	174 case TypeFunc::Control:
a61af66fc99e Initial load duke parents: diff changeset	175 {
a61af66fc99e Initial load duke parents: diff changeset	176 // For Control (fallthrough) and I_O (catch_all_index) we have CatchProj -> Catch -> Proj
a61af66fc99e Initial load duke parents: diff changeset	177 _fallthroughproj = pn;
a61af66fc99e Initial load duke parents: diff changeset	178 DUIterator_Fast jmax, j = pn->fast_outs(jmax);
a61af66fc99e Initial load duke parents: diff changeset	179 const Node *cn = pn->fast_out(j);
a61af66fc99e Initial load duke parents: diff changeset	180 if (cn->is_Catch()) {
a61af66fc99e Initial load duke parents: diff changeset	181 ProjNode *cpn = NULL;
a61af66fc99e Initial load duke parents: diff changeset	182 for (DUIterator_Fast kmax, k = cn->fast_outs(kmax); k < kmax; k++) {
a61af66fc99e Initial load duke parents: diff changeset	183 cpn = cn->fast_out(k)->as_Proj();
a61af66fc99e Initial load duke parents: diff changeset	184 assert(cpn->is_CatchProj(), "must be a CatchProjNode");
a61af66fc99e Initial load duke parents: diff changeset	185 if (cpn->_con == CatchProjNode::fall_through_index)
a61af66fc99e Initial load duke parents: diff changeset	186 _fallthroughcatchproj = cpn;
a61af66fc99e Initial load duke parents: diff changeset	187 else {
a61af66fc99e Initial load duke parents: diff changeset	188 assert(cpn->_con == CatchProjNode::catch_all_index, "must be correct index.");
a61af66fc99e Initial load duke parents: diff changeset	189 _catchallcatchproj = cpn;
a61af66fc99e Initial load duke parents: diff changeset	190 }
a61af66fc99e Initial load duke parents: diff changeset	191 }
a61af66fc99e Initial load duke parents: diff changeset	192 }
a61af66fc99e Initial load duke parents: diff changeset	193 break;
a61af66fc99e Initial load duke parents: diff changeset	194 }
a61af66fc99e Initial load duke parents: diff changeset	195 case TypeFunc::I_O:
a61af66fc99e Initial load duke parents: diff changeset	196 if (pn->_is_io_use)
a61af66fc99e Initial load duke parents: diff changeset	197 _ioproj_catchall = pn;
a61af66fc99e Initial load duke parents: diff changeset	198 else
a61af66fc99e Initial load duke parents: diff changeset	199 _ioproj_fallthrough = pn;
a61af66fc99e Initial load duke parents: diff changeset	200 break;
a61af66fc99e Initial load duke parents: diff changeset	201 case TypeFunc::Memory:
a61af66fc99e Initial load duke parents: diff changeset	202 if (pn->_is_io_use)
a61af66fc99e Initial load duke parents: diff changeset	203 _memproj_catchall = pn;
a61af66fc99e Initial load duke parents: diff changeset	204 else
a61af66fc99e Initial load duke parents: diff changeset	205 _memproj_fallthrough = pn;
a61af66fc99e Initial load duke parents: diff changeset	206 break;
a61af66fc99e Initial load duke parents: diff changeset	207 case TypeFunc::Parms:
a61af66fc99e Initial load duke parents: diff changeset	208 _resproj = pn;
a61af66fc99e Initial load duke parents: diff changeset	209 break;
a61af66fc99e Initial load duke parents: diff changeset	210 default:
a61af66fc99e Initial load duke parents: diff changeset	211 assert(false, "unexpected projection from allocation node.");
a61af66fc99e Initial load duke parents: diff changeset	212 }
a61af66fc99e Initial load duke parents: diff changeset	213 }
a61af66fc99e Initial load duke parents: diff changeset	214
a61af66fc99e Initial load duke parents: diff changeset	215 }
a61af66fc99e Initial load duke parents: diff changeset	216
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	217 // Eliminate a card mark sequence. p2x is a ConvP2XNode
851 fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	218 void PhaseMacroExpand::eliminate_card_mark(Node* p2x) {
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	219 assert(p2x->Opcode() == Op_CastP2X, "ConvP2XNode required");
851 fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	220 if (!UseG1GC) {
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	221 // vanilla/CMS post barrier
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	222 Node *shift = p2x->unique_out();
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	223 Node *addp = shift->unique_out();
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	224 for (DUIterator_Last jmin, j = addp->last_outs(jmin); j >= jmin; --j) {
3282 149bb459be66 7029167: add support for conditional card marks never parents: 2100 diff changeset	225 Node *mem = addp->last_out(j);
149bb459be66 7029167: add support for conditional card marks never parents: 2100 diff changeset	226 if (UseCondCardMark && mem->is_Load()) {
149bb459be66 7029167: add support for conditional card marks never parents: 2100 diff changeset	227 assert(mem->Opcode() == Op_LoadB, "unexpected code shape");
149bb459be66 7029167: add support for conditional card marks never parents: 2100 diff changeset	228 // The load is checking if the card has been written so
149bb459be66 7029167: add support for conditional card marks never parents: 2100 diff changeset	229 // replace it with zero to fold the test.
149bb459be66 7029167: add support for conditional card marks never parents: 2100 diff changeset	230 _igvn.replace_node(mem, intcon(0));
149bb459be66 7029167: add support for conditional card marks never parents: 2100 diff changeset	231 continue;
149bb459be66 7029167: add support for conditional card marks never parents: 2100 diff changeset	232 }
149bb459be66 7029167: add support for conditional card marks never parents: 2100 diff changeset	233 assert(mem->is_Store(), "store required");
149bb459be66 7029167: add support for conditional card marks never parents: 2100 diff changeset	234 _igvn.replace_node(mem, mem->in(MemNode::Memory));
851 fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	235 }
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	236 } else {
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	237 // G1 pre/post barriers
4894 b9bc6cae88f2 7143491: G1 C2 CTW: assert(p2x->outcnt() == 2) failed: expects 2 users: Xor and URShift nodes kvn parents: 4792 diff changeset	238 assert(p2x->outcnt() <= 2, "expects 1 or 2 users: Xor and URShift nodes");
851 fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	239 // It could be only one user, URShift node, in Object.clone() instrinsic
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	240 // but the new allocation is passed to arraycopy stub and it could not
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	241 // be scalar replaced. So we don't check the case.
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	242
4894 b9bc6cae88f2 7143491: G1 C2 CTW: assert(p2x->outcnt() == 2) failed: expects 2 users: Xor and URShift nodes kvn parents: 4792 diff changeset	243 // An other case of only one user (Xor) is when the value check for NULL
b9bc6cae88f2 7143491: G1 C2 CTW: assert(p2x->outcnt() == 2) failed: expects 2 users: Xor and URShift nodes kvn parents: 4792 diff changeset	244 // in G1 post barrier is folded after CCP so the code which used URShift
b9bc6cae88f2 7143491: G1 C2 CTW: assert(p2x->outcnt() == 2) failed: expects 2 users: Xor and URShift nodes kvn parents: 4792 diff changeset	245 // is removed.
b9bc6cae88f2 7143491: G1 C2 CTW: assert(p2x->outcnt() == 2) failed: expects 2 users: Xor and URShift nodes kvn parents: 4792 diff changeset	246
b9bc6cae88f2 7143491: G1 C2 CTW: assert(p2x->outcnt() == 2) failed: expects 2 users: Xor and URShift nodes kvn parents: 4792 diff changeset	247 // Take Region node before eliminating post barrier since it also
b9bc6cae88f2 7143491: G1 C2 CTW: assert(p2x->outcnt() == 2) failed: expects 2 users: Xor and URShift nodes kvn parents: 4792 diff changeset	248 // eliminates CastP2X node when it has only one user.
b9bc6cae88f2 7143491: G1 C2 CTW: assert(p2x->outcnt() == 2) failed: expects 2 users: Xor and URShift nodes kvn parents: 4792 diff changeset	249 Node* this_region = p2x->in(0);
b9bc6cae88f2 7143491: G1 C2 CTW: assert(p2x->outcnt() == 2) failed: expects 2 users: Xor and URShift nodes kvn parents: 4792 diff changeset	250 assert(this_region != NULL, "");
b9bc6cae88f2 7143491: G1 C2 CTW: assert(p2x->outcnt() == 2) failed: expects 2 users: Xor and URShift nodes kvn parents: 4792 diff changeset	251
851 fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	252 // Remove G1 post barrier.
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	253
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	254 // Search for CastP2X->Xor->URShift->Cmp path which
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	255 // checks if the store done to a different from the value's region.
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	256 // And replace Cmp with #0 (false) to collapse G1 post barrier.
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	257 Node* xorx = NULL;
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	258 for (DUIterator_Fast imax, i = p2x->fast_outs(imax); i < imax; i++) {
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	259 Node* u = p2x->fast_out(i);
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	260 if (u->Opcode() == Op_XorX) {
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	261 xorx = u;
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	262 break;
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	263 }
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	264 }
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	265 assert(xorx != NULL, "missing G1 post barrier");
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	266 Node* shift = xorx->unique_out();
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	267 Node* cmpx = shift->unique_out();
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	268 assert(cmpx->is_Cmp() && cmpx->unique_out()->is_Bool() &&
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	269 cmpx->unique_out()->as_Bool()->_test._test == BoolTest::ne,
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	270 "missing region check in G1 post barrier");
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	271 _igvn.replace_node(cmpx, makecon(TypeInt::CC_EQ));
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	272
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	273 // Remove G1 pre barrier.
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	274
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	275 // Search "if (marking != 0)" check and set it to "false".
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	276 // There is no G1 pre barrier if previous stored value is NULL
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	277 // (for example, after initialization).
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	278 if (this_region->is_Region() && this_region->req() == 3) {
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	279 int ind = 1;
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	280 if (!this_region->in(ind)->is_IfFalse()) {
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	281 ind = 2;
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	282 }
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	283 if (this_region->in(ind)->is_IfFalse()) {
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	284 Node* bol = this_region->in(ind)->in(0)->in(1);
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	285 assert(bol->is_Bool(), "");
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	286 cmpx = bol->in(1);
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	287 if (bol->as_Bool()->_test._test == BoolTest::ne &&
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	288 cmpx->is_Cmp() && cmpx->in(2) == intcon(0) &&
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	289 cmpx->in(1)->is_Load()) {
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	290 Node* adr = cmpx->in(1)->as_Load()->in(MemNode::Address);
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	291 const int marking_offset = in_bytes(JavaThread::satb_mark_queue_offset() +
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	292 PtrQueue::byte_offset_of_active());
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	293 if (adr->is_AddP() && adr->in(AddPNode::Base) == top() &&
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	294 adr->in(AddPNode::Address)->Opcode() == Op_ThreadLocal &&
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	295 adr->in(AddPNode::Offset) == MakeConX(marking_offset)) {
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	296 _igvn.replace_node(cmpx, makecon(TypeInt::CC_EQ));
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	297 }
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	298 }
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	299 }
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	300 }
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	301 // Now CastP2X can be removed since it is used only on dead path
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	302 // which currently still alive until igvn optimize it.
4894 b9bc6cae88f2 7143491: G1 C2 CTW: assert(p2x->outcnt() == 2) failed: expects 2 users: Xor and URShift nodes kvn parents: 4792 diff changeset	303 assert(p2x->outcnt() == 0 \|\| p2x->unique_out()->Opcode() == Op_URShiftX, "");
851 fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	304 _igvn.replace_node(p2x, top());
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	305 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	306 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	307
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	308 // Search for a memory operation for the specified memory slice.
253 b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	309 static Node scan_mem_chain(Node mem, int alias_idx, int offset, Node start_mem, Node alloc, PhaseGVN *phase) {
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	310 Node *orig_mem = mem;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	311 Node *alloc_mem = alloc->in(TypeFunc::Memory);
253 b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	312 const TypeOopPtr *tinst = phase->C->get_adr_type(alias_idx)->isa_oopptr();
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	313 while (true) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	314 if (mem == alloc_mem \|\| mem == start_mem ) {
605 98cb887364d3 6810672: Comment typos twisti parents: 601 diff changeset	315 return mem; // hit one of our sentinels
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	316 } else if (mem->is_MergeMem()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	317 mem = mem->as_MergeMem()->memory_at(alias_idx);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	318 } else if (mem->is_Proj() && mem->as_Proj()->_con == TypeFunc::Memory) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	319 Node *in = mem->in(0);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	320 // we can safely skip over safepoints, calls, locks and membars because we
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	321 // already know that the object is safe to eliminate.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	322 if (in->is_Initialize() && in->as_Initialize()->allocation() == alloc) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	323 return in;
253 b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	324 } else if (in->is_Call()) {
b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	325 CallNode *call = in->as_Call();
b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	326 if (!call->may_modify(tinst, phase)) {
b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	327 mem = call->in(TypeFunc::Memory);
b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	328 }
b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	329 mem = in->in(TypeFunc::Memory);
b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	330 } else if (in->is_MemBar()) {
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	331 mem = in->in(TypeFunc::Memory);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	332 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	333 assert(false, "unexpected projection");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	334 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	335 } else if (mem->is_Store()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	336 const TypePtr* atype = mem->as_Store()->adr_type();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	337 int adr_idx = Compile::current()->get_alias_index(atype);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	338 if (adr_idx == alias_idx) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	339 assert(atype->isa_oopptr(), "address type must be oopptr");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	340 int adr_offset = atype->offset();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	341 uint adr_iid = atype->is_oopptr()->instance_id();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	342 // Array elements references have the same alias_idx
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	343 // but different offset and different instance_id.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	344 if (adr_offset == offset && adr_iid == alloc->_idx)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	345 return mem;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	346 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	347 assert(adr_idx == Compile::AliasIdxRaw, "address must match or be raw");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	348 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	349 mem = mem->in(MemNode::Memory);
1100 f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	350 } else if (mem->is_ClearArray()) {
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	351 if (!ClearArrayNode::step_through(&mem, alloc->_idx, phase)) {
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	352 // Can not bypass initialization of the instance
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	353 // we are looking.
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	354 debug_only(intptr_t offset;)
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	355 assert(alloc == AllocateNode::Ideal_allocation(mem->in(3), phase, offset), "sanity");
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	356 InitializeNode* init = alloc->as_Allocate()->initialization();
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	357 // We are looking for stored value, return Initialize node
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	358 // or memory edge from Allocate node.
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	359 if (init != NULL)
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	360 return init;
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	361 else
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	362 return alloc->in(TypeFunc::Memory); // It will produce zero value (see callers).
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	363 }
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	364 // Otherwise skip it (the call updated 'mem' value).
584 49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	365 } else if (mem->Opcode() == Op_SCMemProj) {
7637 b30b3c2a0cf2 6896617: Optimize sun.nio.cs.ISO_8859_1$Encode.encodeArrayLoop() on x86 kvn parents: 7196 diff changeset	366 mem = mem->in(0);
b30b3c2a0cf2 6896617: Optimize sun.nio.cs.ISO_8859_1$Encode.encodeArrayLoop() on x86 kvn parents: 7196 diff changeset	367 Node* adr = NULL;
b30b3c2a0cf2 6896617: Optimize sun.nio.cs.ISO_8859_1$Encode.encodeArrayLoop() on x86 kvn parents: 7196 diff changeset	368 if (mem->is_LoadStore()) {
b30b3c2a0cf2 6896617: Optimize sun.nio.cs.ISO_8859_1$Encode.encodeArrayLoop() on x86 kvn parents: 7196 diff changeset	369 adr = mem->in(MemNode::Address);
b30b3c2a0cf2 6896617: Optimize sun.nio.cs.ISO_8859_1$Encode.encodeArrayLoop() on x86 kvn parents: 7196 diff changeset	370 } else {
b30b3c2a0cf2 6896617: Optimize sun.nio.cs.ISO_8859_1$Encode.encodeArrayLoop() on x86 kvn parents: 7196 diff changeset	371 assert(mem->Opcode() == Op_EncodeISOArray, "sanity");
b30b3c2a0cf2 6896617: Optimize sun.nio.cs.ISO_8859_1$Encode.encodeArrayLoop() on x86 kvn parents: 7196 diff changeset	372 adr = mem->in(3); // Destination array
b30b3c2a0cf2 6896617: Optimize sun.nio.cs.ISO_8859_1$Encode.encodeArrayLoop() on x86 kvn parents: 7196 diff changeset	373 }
b30b3c2a0cf2 6896617: Optimize sun.nio.cs.ISO_8859_1$Encode.encodeArrayLoop() on x86 kvn parents: 7196 diff changeset	374 const TypePtr* atype = adr->bottom_type()->is_ptr();
584 49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	375 int adr_idx = Compile::current()->get_alias_index(atype);
49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	376 if (adr_idx == alias_idx) {
49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	377 assert(false, "Object is not scalar replaceable if a LoadStore node access its field");
49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	378 return NULL;
49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	379 }
7637 b30b3c2a0cf2 6896617: Optimize sun.nio.cs.ISO_8859_1$Encode.encodeArrayLoop() on x86 kvn parents: 7196 diff changeset	380 mem = mem->in(MemNode::Memory);
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	381 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	382 return mem;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	383 }
247 02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	384 assert(mem != orig_mem, "dead memory loop");
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	385 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	386 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	387
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	388 //
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	389 // Given a Memory Phi, compute a value Phi containing the values from stores
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	390 // on the input paths.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	391 // Note: this function is recursive, its depth is limied by the "level" argument
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	392 // Returns the computed Phi, or NULL if it cannot compute it.
247 02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	393 Node PhaseMacroExpand::value_from_mem_phi(Node mem, BasicType ft, const Type phi_type, const TypeOopPtr adr_t, Node alloc, Node_Stack value_phis, int level) {
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	394 assert(mem->is_Phi(), "sanity");
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	395 int alias_idx = C->get_alias_index(adr_t);
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	396 int offset = adr_t->offset();
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	397 int instance_id = adr_t->instance_id();
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	398
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	399 // Check if an appropriate value phi already exists.
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	400 Node* region = mem->in(0);
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	401 for (DUIterator_Fast kmax, k = region->fast_outs(kmax); k < kmax; k++) {
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	402 Node* phi = region->fast_out(k);
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	403 if (phi->is_Phi() && phi != mem &&
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	404 phi->as_Phi()->is_same_inst_field(phi_type, instance_id, alias_idx, offset)) {
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	405 return phi;
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	406 }
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	407 }
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	408 // Check if an appropriate new value phi already exists.
3788 e3cbc9ddd434 7044738: Loop unroll optimization causes incorrect result kvn parents: 3754 diff changeset	409 Node* new_phi = value_phis->find(mem->_idx);
e3cbc9ddd434 7044738: Loop unroll optimization causes incorrect result kvn parents: 3754 diff changeset	410 if (new_phi != NULL)
e3cbc9ddd434 7044738: Loop unroll optimization causes incorrect result kvn parents: 3754 diff changeset	411 return new_phi;
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	412
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	413 if (level <= 0) {
253 b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	414 return NULL; // Give up: phi tree too deep
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	415 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	416 Node *start_mem = C->start()->proj_out(TypeFunc::Memory);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	417 Node *alloc_mem = alloc->in(TypeFunc::Memory);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	418
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	419 uint length = mem->req();
6197 d2a62e0f25eb 6995781: Native Memory Tracking (Phase 1) zgu parents: 6144 diff changeset	420 GrowableArray <Node *> values(length, length, NULL, false);
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	421
247 02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	422 // create a new Phi for the value
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	423 PhiNode *phi = new (C) PhiNode(mem->in(0), phi_type, NULL, instance_id, alias_idx, offset);
247 02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	424 transform_later(phi);
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	425 value_phis->push(phi, mem->_idx);
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	426
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	427 for (uint j = 1; j < length; j++) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	428 Node *in = mem->in(j);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	429 if (in == NULL \|\| in->is_top()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	430 values.at_put(j, in);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	431 } else {
253 b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	432 Node *val = scan_mem_chain(in, alias_idx, offset, start_mem, alloc, &_igvn);
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	433 if (val == start_mem \|\| val == alloc_mem) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	434 // hit a sentinel, return appropriate 0 value
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	435 values.at_put(j, _igvn.zerocon(ft));
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	436 continue;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	437 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	438 if (val->is_Initialize()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	439 val = val->as_Initialize()->find_captured_store(offset, type2aelembytes(ft), &_igvn);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	440 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	441 if (val == NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	442 return NULL; // can't find a value on this path
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	443 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	444 if (val == mem) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	445 values.at_put(j, mem);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	446 } else if (val->is_Store()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	447 values.at_put(j, val->in(MemNode::ValueIn));
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	448 } else if(val->is_Proj() && val->in(0) == alloc) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	449 values.at_put(j, _igvn.zerocon(ft));
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	450 } else if (val->is_Phi()) {
247 02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	451 val = value_from_mem_phi(val, ft, phi_type, adr_t, alloc, value_phis, level-1);
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	452 if (val == NULL) {
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	453 return NULL;
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	454 }
247 02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	455 values.at_put(j, val);
584 49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	456 } else if (val->Opcode() == Op_SCMemProj) {
7637 b30b3c2a0cf2 6896617: Optimize sun.nio.cs.ISO_8859_1$Encode.encodeArrayLoop() on x86 kvn parents: 7196 diff changeset	457 assert(val->in(0)->is_LoadStore() \|\| val->in(0)->Opcode() == Op_EncodeISOArray, "sanity");
584 49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	458 assert(false, "Object is not scalar replaceable if a LoadStore node access its field");
49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	459 return NULL;
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	460 } else {
584 49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	461 #ifdef ASSERT
49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	462 val->dump();
253 b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	463 assert(false, "unknown node on this path");
584 49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	464 #endif
253 b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	465 return NULL; // unknown node on this path
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	466 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	467 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	468 }
247 02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	469 // Set Phi's inputs
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	470 for (uint j = 1; j < length; j++) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	471 if (values.at(j) == mem) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	472 phi->init_req(j, phi);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	473 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	474 phi->init_req(j, values.at(j));
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	475 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	476 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	477 return phi;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	478 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	479
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	480 // Search the last value stored into the object's field.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	481 Node PhaseMacroExpand::value_from_mem(Node sfpt_mem, BasicType ft, const Type ftype, const TypeOopPtr adr_t, Node *alloc) {
223 1dd146f17531 6716441: error in meet with +DoEscapeAnalysis kvn parents: 221 diff changeset	482 assert(adr_t->is_known_instance_field(), "instance required");
1dd146f17531 6716441: error in meet with +DoEscapeAnalysis kvn parents: 221 diff changeset	483 int instance_id = adr_t->instance_id();
1dd146f17531 6716441: error in meet with +DoEscapeAnalysis kvn parents: 221 diff changeset	484 assert((uint)instance_id == alloc->_idx, "wrong allocation");
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	485
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	486 int alias_idx = C->get_alias_index(adr_t);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	487 int offset = adr_t->offset();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	488 Node *start_mem = C->start()->proj_out(TypeFunc::Memory);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	489 Node *alloc_ctrl = alloc->in(TypeFunc::Control);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	490 Node *alloc_mem = alloc->in(TypeFunc::Memory);
247 02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	491 Arena *a = Thread::current()->resource_area();
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	492 VectorSet visited(a);
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	493
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	494
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	495 bool done = sfpt_mem == alloc_mem;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	496 Node *mem = sfpt_mem;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	497 while (!done) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	498 if (visited.test_set(mem->_idx)) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	499 return NULL; // found a loop, give up
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	500 }
253 b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	501 mem = scan_mem_chain(mem, alias_idx, offset, start_mem, alloc, &_igvn);
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	502 if (mem == start_mem \|\| mem == alloc_mem) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	503 done = true; // hit a sentinel, return appropriate 0 value
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	504 } else if (mem->is_Initialize()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	505 mem = mem->as_Initialize()->find_captured_store(offset, type2aelembytes(ft), &_igvn);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	506 if (mem == NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	507 done = true; // Something go wrong.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	508 } else if (mem->is_Store()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	509 const TypePtr* atype = mem->as_Store()->adr_type();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	510 assert(C->get_alias_index(atype) == Compile::AliasIdxRaw, "store is correct memory slice");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	511 done = true;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	512 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	513 } else if (mem->is_Store()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	514 const TypeOopPtr* atype = mem->as_Store()->adr_type()->isa_oopptr();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	515 assert(atype != NULL, "address type must be oopptr");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	516 assert(C->get_alias_index(atype) == alias_idx &&
223 1dd146f17531 6716441: error in meet with +DoEscapeAnalysis kvn parents: 221 diff changeset	517 atype->is_known_instance_field() && atype->offset() == offset &&
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	518 atype->instance_id() == instance_id, "store is correct memory slice");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	519 done = true;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	520 } else if (mem->is_Phi()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	521 // try to find a phi's unique input
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	522 Node *unique_input = NULL;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	523 Node *top = C->top();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	524 for (uint i = 1; i < mem->req(); i++) {
253 b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	525 Node *n = scan_mem_chain(mem->in(i), alias_idx, offset, start_mem, alloc, &_igvn);
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	526 if (n == NULL \|\| n == top \|\| n == mem) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	527 continue;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	528 } else if (unique_input == NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	529 unique_input = n;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	530 } else if (unique_input != n) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	531 unique_input = top;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	532 break;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	533 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	534 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	535 if (unique_input != NULL && unique_input != top) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	536 mem = unique_input;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	537 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	538 done = true;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	539 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	540 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	541 assert(false, "unexpected node");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	542 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	543 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	544 if (mem != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	545 if (mem == start_mem \|\| mem == alloc_mem) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	546 // hit a sentinel, return appropriate 0 value
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	547 return _igvn.zerocon(ft);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	548 } else if (mem->is_Store()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	549 return mem->in(MemNode::ValueIn);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	550 } else if (mem->is_Phi()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	551 // attempt to produce a Phi reflecting the values on the input paths of the Phi
247 02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	552 Node_Stack value_phis(a, 8);
253 b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	553 Node * phi = value_from_mem_phi(mem, ft, ftype, adr_t, alloc, &value_phis, ValueSearchLimit);
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	554 if (phi != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	555 return phi;
247 02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	556 } else {
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	557 // Kill all new Phis
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	558 while(value_phis.is_nonempty()) {
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	559 Node* n = value_phis.node();
1621 6027dddc26c6 6677629: PhaseIterGVN::subsume_node() should call hash_delete() and add_users_to_worklist() kvn parents: 1609 diff changeset	560 _igvn.replace_node(n, C->top());
247 02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	561 value_phis.pop();
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	562 }
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	563 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	564 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	565 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	566 // Something go wrong.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	567 return NULL;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	568 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	569
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	570 // Check the possibility of scalar replacement.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	571 bool PhaseMacroExpand::can_eliminate_allocation(AllocateNode alloc, GrowableArray <SafePointNode >& safepoints) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	572 // Scan the uses of the allocation to check for anything that would
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	573 // prevent us from eliminating it.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	574 NOT_PRODUCT( const char* fail_eliminate = NULL; )
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	575 DEBUG_ONLY( Node* disq_node = NULL; )
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	576 bool can_eliminate = true;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	577
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	578 Node* res = alloc->result_cast();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	579 const TypeOopPtr* res_type = NULL;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	580 if (res == NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	581 // All users were eliminated.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	582 } else if (!res->is_CheckCastPP()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	583 NOT_PRODUCT(fail_eliminate = "Allocation does not have unique CheckCastPP";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	584 can_eliminate = false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	585 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	586 res_type = _igvn.type(res)->isa_oopptr();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	587 if (res_type == NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	588 NOT_PRODUCT(fail_eliminate = "Neither instance or array allocation";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	589 can_eliminate = false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	590 } else if (res_type->isa_aryptr()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	591 int length = alloc->in(AllocateNode::ALength)->find_int_con(-1);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	592 if (length < 0) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	593 NOT_PRODUCT(fail_eliminate = "Array's size is not constant";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	594 can_eliminate = false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	595 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	596 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	597 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	598
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	599 if (can_eliminate && res != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	600 for (DUIterator_Fast jmax, j = res->fast_outs(jmax);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	601 j < jmax && can_eliminate; j++) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	602 Node* use = res->fast_out(j);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	603
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	604 if (use->is_AddP()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	605 const TypePtr* addp_type = _igvn.type(use)->is_ptr();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	606 int offset = addp_type->offset();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	607
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	608 if (offset == Type::OffsetTop \|\| offset == Type::OffsetBot) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	609 NOT_PRODUCT(fail_eliminate = "Undefined field referrence";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	610 can_eliminate = false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	611 break;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	612 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	613 for (DUIterator_Fast kmax, k = use->fast_outs(kmax);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	614 k < kmax && can_eliminate; k++) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	615 Node* n = use->fast_out(k);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	616 if (!n->is_Store() && n->Opcode() != Op_CastP2X) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	617 DEBUG_ONLY(disq_node = n;)
253 b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	618 if (n->is_Load() \|\| n->is_LoadStore()) {
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	619 NOT_PRODUCT(fail_eliminate = "Field load";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	620 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	621 NOT_PRODUCT(fail_eliminate = "Not store field referrence";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	622 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	623 can_eliminate = false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	624 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	625 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	626 } else if (use->is_SafePoint()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	627 SafePointNode* sfpt = use->as_SafePoint();
168 7793bd37a336 6705887: Compressed Oops: generate x64 addressing and implicit null checks with narrow oops kvn parents: 163 diff changeset	628 if (sfpt->is_Call() && sfpt->as_Call()->has_non_debug_use(res)) {
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	629 // Object is passed as argument.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	630 DEBUG_ONLY(disq_node = use;)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	631 NOT_PRODUCT(fail_eliminate = "Object is passed as argument";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	632 can_eliminate = false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	633 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	634 Node* sfptMem = sfpt->memory();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	635 if (sfptMem == NULL \|\| sfptMem->is_top()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	636 DEBUG_ONLY(disq_node = use;)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	637 NOT_PRODUCT(fail_eliminate = "NULL or TOP memory";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	638 can_eliminate = false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	639 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	640 safepoints.append_if_missing(sfpt);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	641 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	642 } else if (use->Opcode() != Op_CastP2X) { // CastP2X is used by card mark
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	643 if (use->is_Phi()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	644 if (use->outcnt() == 1 && use->unique_out()->Opcode() == Op_Return) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	645 NOT_PRODUCT(fail_eliminate = "Object is return value";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	646 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	647 NOT_PRODUCT(fail_eliminate = "Object is referenced by Phi";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	648 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	649 DEBUG_ONLY(disq_node = use;)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	650 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	651 if (use->Opcode() == Op_Return) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	652 NOT_PRODUCT(fail_eliminate = "Object is return value";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	653 }else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	654 NOT_PRODUCT(fail_eliminate = "Object is referenced by node";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	655 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	656 DEBUG_ONLY(disq_node = use;)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	657 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	658 can_eliminate = false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	659 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	660 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	661 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	662
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	663 #ifndef PRODUCT
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	664 if (PrintEliminateAllocations) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	665 if (can_eliminate) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	666 tty->print("Scalar ");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	667 if (res == NULL)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	668 alloc->dump();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	669 else
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	670 res->dump();
10278 6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	671 } else if (alloc->_is_scalar_replaceable) {
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	672 tty->print("NotScalar (%s)", fail_eliminate);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	673 if (res == NULL)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	674 alloc->dump();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	675 else
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	676 res->dump();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	677 #ifdef ASSERT
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	678 if (disq_node != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	679 tty->print(" >>>> ");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	680 disq_node->dump();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	681 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	682 #endif /ASSERT/
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	683 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	684 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	685 #endif
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	686 return can_eliminate;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	687 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	688
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	689 // Do scalar replacement.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	690 bool PhaseMacroExpand::scalar_replacement(AllocateNode alloc, GrowableArray <SafePointNode >& safepoints) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	691 GrowableArray <SafePointNode *> safepoints_done;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	692
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	693 ciKlass* klass = NULL;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	694 ciInstanceKlass* iklass = NULL;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	695 int nfields = 0;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	696 int array_base;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	697 int element_size;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	698 BasicType basic_elem_type;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	699 ciType* elem_type;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	700
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	701 Node* res = alloc->result_cast();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	702 const TypeOopPtr* res_type = NULL;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	703 if (res != NULL) { // Could be NULL when there are no users
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	704 res_type = _igvn.type(res)->isa_oopptr();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	705 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	706
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	707 if (res != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	708 klass = res_type->klass();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	709 if (res_type->isa_instptr()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	710 // find the fields of the class which will be needed for safepoint debug information
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	711 assert(klass->is_instance_klass(), "must be an instance klass.");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	712 iklass = klass->as_instance_klass();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	713 nfields = iklass->nof_nonstatic_fields();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	714 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	715 // find the array's elements which will be needed for safepoint debug information
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	716 nfields = alloc->in(AllocateNode::ALength)->find_int_con(-1);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	717 assert(klass->is_array_klass() && nfields >= 0, "must be an array klass.");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	718 elem_type = klass->as_array_klass()->element_type();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	719 basic_elem_type = elem_type->basic_type();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	720 array_base = arrayOopDesc::base_offset_in_bytes(basic_elem_type);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	721 element_size = type2aelembytes(basic_elem_type);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	722 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	723 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	724 //
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	725 // Process the safepoint uses
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	726 //
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	727 while (safepoints.length() > 0) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	728 SafePointNode* sfpt = safepoints.pop();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	729 Node* mem = sfpt->memory();
12158 766fac3395d6 8012972: Incremental Inlining should support scalar replaced object in debug info kvn parents: 10279 diff changeset	730 assert(sfpt->jvms() != NULL, "missed JVMS");
766fac3395d6 8012972: Incremental Inlining should support scalar replaced object in debug info kvn parents: 10279 diff changeset	731 // Fields of scalar objs are referenced only at the end
766fac3395d6 8012972: Incremental Inlining should support scalar replaced object in debug info kvn parents: 10279 diff changeset	732 // of regular debuginfo at the last (youngest) JVMS.
766fac3395d6 8012972: Incremental Inlining should support scalar replaced object in debug info kvn parents: 10279 diff changeset	733 // Record relative start index.
766fac3395d6 8012972: Incremental Inlining should support scalar replaced object in debug info kvn parents: 10279 diff changeset	734 uint first_ind = (sfpt->req() - sfpt->jvms()->scloff());
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	735 SafePointScalarObjectNode* sobj = new (C) SafePointScalarObjectNode(res_type,
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	736 #ifdef ASSERT
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	737 alloc,
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	738 #endif
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	739 first_ind, nfields);
4115 1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	740 sobj->init_req(0, C->root());
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	741 transform_later(sobj);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	742
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	743 // Scan object's fields adding an input to the safepoint for each field.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	744 for (int j = 0; j < nfields; j++) {
306 af945ba2e739 6741738: TypePtr::add_offset() set incorrect offset when the add overflows kvn parents: 253 diff changeset	745 intptr_t offset;
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	746 ciField* field = NULL;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	747 if (iklass != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	748 field = iklass->nonstatic_field_at(j);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	749 offset = field->offset();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	750 elem_type = field->type();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	751 basic_elem_type = field->layout_type();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	752 } else {
306 af945ba2e739 6741738: TypePtr::add_offset() set incorrect offset when the add overflows kvn parents: 253 diff changeset	753 offset = array_base + j * (intptr_t)element_size;
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	754 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	755
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	756 const Type *field_type;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	757 // The next code is taken from Parse::do_get_xxx().
124 b130b98db9cf 6689060: Escape Analysis does not work with Compressed Oops kvn parents: 113 diff changeset	758 if (basic_elem_type == T_OBJECT \|\| basic_elem_type == T_ARRAY) {
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	759 if (!elem_type->is_loaded()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	760 field_type = TypeInstPtr::BOTTOM;
1682 e5dfb3ccb88b 6969569: assert(is_static() && is_constant()) failed: illegal call to constant_value() kvn parents: 1621 diff changeset	761 } else if (field != NULL && field->is_constant() && field->is_static()) {
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	762 // This can happen if the constant oop is non-perm.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	763 ciObject* con = field->constant_value().as_object();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	764 // Do not "join" in the previous type; it doesn't add value,
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	765 // and may yield a vacuous result if the field is of interface type.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	766 field_type = TypeOopPtr::make_from_constant(con)->isa_oopptr();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	767 assert(field_type != NULL, "field singleton type must be consistent");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	768 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	769 field_type = TypeOopPtr::make_from_klass(elem_type->as_klass());
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	770 }
124 b130b98db9cf 6689060: Escape Analysis does not work with Compressed Oops kvn parents: 113 diff changeset	771 if (UseCompressedOops) {
221 1e026f8da827 6710487: More than half of JDI Regression tests hang with COOPs in -Xcomp mode kvn parents: 216 diff changeset	772 field_type = field_type->make_narrowoop();
124 b130b98db9cf 6689060: Escape Analysis does not work with Compressed Oops kvn parents: 113 diff changeset	773 basic_elem_type = T_NARROWOOP;
b130b98db9cf 6689060: Escape Analysis does not work with Compressed Oops kvn parents: 113 diff changeset	774 }
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	775 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	776 field_type = Type::get_const_basic_type(basic_elem_type);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	777 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	778
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	779 const TypeOopPtr *field_addr_type = res_type->add_offset(offset)->isa_oopptr();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	780
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	781 Node *field_val = value_from_mem(mem, basic_elem_type, field_type, field_addr_type, alloc);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	782 if (field_val == NULL) {
4115 1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	783 // We weren't able to find a value for this field,
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	784 // give up on eliminating this allocation.
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	785
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	786 // Remove any extra entries we added to the safepoint.
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	787 uint last = sfpt->req() - 1;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	788 for (int k = 0; k < j; k++) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	789 sfpt->del_req(last--);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	790 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	791 // rollback processed safepoints
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	792 while (safepoints_done.length() > 0) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	793 SafePointNode* sfpt_done = safepoints_done.pop();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	794 // remove any extra entries we added to the safepoint
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	795 last = sfpt_done->req() - 1;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	796 for (int k = 0; k < nfields; k++) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	797 sfpt_done->del_req(last--);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	798 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	799 JVMState *jvms = sfpt_done->jvms();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	800 jvms->set_endoff(sfpt_done->req());
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	801 // Now make a pass over the debug information replacing any references
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	802 // to SafePointScalarObjectNode with the allocated object.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	803 int start = jvms->debug_start();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	804 int end = jvms->debug_end();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	805 for (int i = start; i < end; i++) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	806 if (sfpt_done->in(i)->is_SafePointScalarObject()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	807 SafePointScalarObjectNode* scobj = sfpt_done->in(i)->as_SafePointScalarObject();
12158 766fac3395d6 8012972: Incremental Inlining should support scalar replaced object in debug info kvn parents: 10279 diff changeset	808 if (scobj->first_index(jvms) == sfpt_done->req() &&
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	809 scobj->n_fields() == (uint)nfields) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	810 assert(scobj->alloc() == alloc, "sanity");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	811 sfpt_done->set_req(i, res);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	812 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	813 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	814 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	815 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	816 #ifndef PRODUCT
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	817 if (PrintEliminateAllocations) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	818 if (field != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	819 tty->print("=== At SafePoint node %d can't find value of Field: ",
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	820 sfpt->_idx);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	821 field->print();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	822 int field_idx = C->get_alias_index(field_addr_type);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	823 tty->print(" (alias_idx=%d)", field_idx);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	824 } else { // Array's element
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	825 tty->print("=== At SafePoint node %d can't find value of array element [%d]",
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	826 sfpt->_idx, j);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	827 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	828 tty->print(", which prevents elimination of: ");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	829 if (res == NULL)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	830 alloc->dump();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	831 else
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	832 res->dump();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	833 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	834 #endif
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	835 return false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	836 }
124 b130b98db9cf 6689060: Escape Analysis does not work with Compressed Oops kvn parents: 113 diff changeset	837 if (UseCompressedOops && field_type->isa_narrowoop()) {
b130b98db9cf 6689060: Escape Analysis does not work with Compressed Oops kvn parents: 113 diff changeset	838 // Enable "DecodeN(EncodeP(Allocate)) --> Allocate" transformation
b130b98db9cf 6689060: Escape Analysis does not work with Compressed Oops kvn parents: 113 diff changeset	839 // to be able scalar replace the allocation.
221 1e026f8da827 6710487: More than half of JDI Regression tests hang with COOPs in -Xcomp mode kvn parents: 216 diff changeset	840 if (field_val->is_EncodeP()) {
1e026f8da827 6710487: More than half of JDI Regression tests hang with COOPs in -Xcomp mode kvn parents: 216 diff changeset	841 field_val = field_val->in(1);
1e026f8da827 6710487: More than half of JDI Regression tests hang with COOPs in -Xcomp mode kvn parents: 216 diff changeset	842 } else {
10279 70120f47d403 8014189: JVM crash with SEGV in ConnectionGraph::record_for_escape_analysis() kvn parents: 10278 diff changeset	843 field_val = transform_later(new (C) DecodeNNode(field_val, field_val->get_ptr_type()));
221 1e026f8da827 6710487: More than half of JDI Regression tests hang with COOPs in -Xcomp mode kvn parents: 216 diff changeset	844 }
124 b130b98db9cf 6689060: Escape Analysis does not work with Compressed Oops kvn parents: 113 diff changeset	845 }
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	846 sfpt->add_req(field_val);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	847 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	848 JVMState *jvms = sfpt->jvms();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	849 jvms->set_endoff(sfpt->req());
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	850 // Now make a pass over the debug information replacing any references
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	851 // to the allocated object with "sobj"
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	852 int start = jvms->debug_start();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	853 int end = jvms->debug_end();
10278 6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	854 sfpt->replace_edges_in_range(res, sobj, start, end);
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	855 safepoints_done.append_if_missing(sfpt); // keep it for rollback
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	856 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	857 return true;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	858 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	859
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	860 // Process users of eliminated allocation.
10278 6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	861 void PhaseMacroExpand::process_users_of_allocation(CallNode *alloc) {
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	862 Node* res = alloc->result_cast();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	863 if (res != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	864 for (DUIterator_Last jmin, j = res->last_outs(jmin); j >= jmin; ) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	865 Node *use = res->last_out(j);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	866 uint oc1 = res->outcnt();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	867
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	868 if (use->is_AddP()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	869 for (DUIterator_Last kmin, k = use->last_outs(kmin); k >= kmin; ) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	870 Node *n = use->last_out(k);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	871 uint oc2 = use->outcnt();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	872 if (n->is_Store()) {
1100 f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	873 #ifdef ASSERT
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	874 // Verify that there is no dependent MemBarVolatile nodes,
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	875 // they should be removed during IGVN, see MemBarNode::Ideal().
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	876 for (DUIterator_Fast pmax, p = n->fast_outs(pmax);
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	877 p < pmax; p++) {
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	878 Node* mb = n->fast_out(p);
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	879 assert(mb->is_Initialize() \|\| !mb->is_MemBar() \|\|
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	880 mb->req() <= MemBarNode::Precedent \|\|
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	881 mb->in(MemBarNode::Precedent) != n,
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	882 "MemBarVolatile should be eliminated for non-escaping object");
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	883 }
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	884 #endif
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	885 _igvn.replace_node(n, n->in(MemNode::Memory));
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	886 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	887 eliminate_card_mark(n);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	888 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	889 k -= (oc2 - use->outcnt());
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	890 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	891 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	892 eliminate_card_mark(use);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	893 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	894 j -= (oc1 - res->outcnt());
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	895 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	896 assert(res->outcnt() == 0, "all uses of allocated objects must be deleted");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	897 _igvn.remove_dead_node(res);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	898 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	899
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	900 //
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	901 // Process other users of allocation's projections
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	902 //
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	903 if (_resproj != NULL && _resproj->outcnt() != 0) {
10278 6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	904 // First disconnect stores captured by Initialize node.
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	905 // If Initialize node is eliminated first in the following code,
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	906 // it will kill such stores and DUIterator_Last will assert.
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	907 for (DUIterator_Fast jmax, j = _resproj->fast_outs(jmax); j < jmax; j++) {
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	908 Node *use = _resproj->fast_out(j);
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	909 if (use->is_AddP()) {
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	910 // raw memory addresses used only by the initialization
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	911 _igvn.replace_node(use, C->top());
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	912 --j; --jmax;
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	913 }
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	914 }
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	915 for (DUIterator_Last jmin, j = _resproj->last_outs(jmin); j >= jmin; ) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	916 Node *use = _resproj->last_out(j);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	917 uint oc1 = _resproj->outcnt();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	918 if (use->is_Initialize()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	919 // Eliminate Initialize node.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	920 InitializeNode *init = use->as_Initialize();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	921 assert(init->outcnt() <= 2, "only a control and memory projection expected");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	922 Node *ctrl_proj = init->proj_out(TypeFunc::Control);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	923 if (ctrl_proj != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	924 assert(init->in(TypeFunc::Control) == _fallthroughcatchproj, "allocation control projection");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	925 _igvn.replace_node(ctrl_proj, _fallthroughcatchproj);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	926 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	927 Node *mem_proj = init->proj_out(TypeFunc::Memory);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	928 if (mem_proj != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	929 Node *mem = init->in(TypeFunc::Memory);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	930 #ifdef ASSERT
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	931 if (mem->is_MergeMem()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	932 assert(mem->in(TypeFunc::Memory) == _memproj_fallthrough, "allocation memory projection");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	933 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	934 assert(mem == _memproj_fallthrough, "allocation memory projection");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	935 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	936 #endif
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	937 _igvn.replace_node(mem_proj, mem);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	938 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	939 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	940 assert(false, "only Initialize or AddP expected");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	941 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	942 j -= (oc1 - _resproj->outcnt());
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	943 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	944 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	945 if (_fallthroughcatchproj != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	946 _igvn.replace_node(_fallthroughcatchproj, alloc->in(TypeFunc::Control));
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	947 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	948 if (_memproj_fallthrough != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	949 _igvn.replace_node(_memproj_fallthrough, alloc->in(TypeFunc::Memory));
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	950 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	951 if (_memproj_catchall != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	952 _igvn.replace_node(_memproj_catchall, C->top());
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	953 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	954 if (_ioproj_fallthrough != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	955 _igvn.replace_node(_ioproj_fallthrough, alloc->in(TypeFunc::I_O));
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	956 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	957 if (_ioproj_catchall != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	958 _igvn.replace_node(_ioproj_catchall, C->top());
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	959 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	960 if (_catchallcatchproj != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	961 _igvn.replace_node(_catchallcatchproj, C->top());
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	962 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	963 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	964
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	965 bool PhaseMacroExpand::eliminate_allocate_node(AllocateNode *alloc) {
10278 6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	966 if (!EliminateAllocations \|\| !alloc->_is_non_escaping) {
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	967 return false;
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	968 }
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	969 Node* klass = alloc->in(AllocateNode::KlassNode);
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	970 const TypeKlassPtr* tklass = _igvn.type(klass)->is_klassptr();
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	971 Node* res = alloc->result_cast();
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	972 // Eliminate boxing allocations which are not used
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	973 // regardless scalar replacable status.
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	974 bool boxing_alloc = C->eliminate_boxing() &&
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	975 tklass->klass()->is_instance_klass() &&
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	976 tklass->klass()->as_instance_klass()->is_box_klass();
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	977 if (!alloc->_is_scalar_replaceable && (!boxing_alloc \|\| (res != NULL))) {
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	978 return false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	979 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	980
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	981 extract_call_projections(alloc);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	982
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	983 GrowableArray <SafePointNode *> safepoints;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	984 if (!can_eliminate_allocation(alloc, safepoints)) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	985 return false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	986 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	987
10278 6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	988 if (!alloc->_is_scalar_replaceable) {
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	989 assert(res == NULL, "sanity");
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	990 // We can only eliminate allocation if all debug info references
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	991 // are already replaced with SafePointScalarObject because
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	992 // we can't search for a fields value without instance_id.
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	993 if (safepoints.length() > 0) {
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	994 return false;
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	995 }
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	996 }
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	997
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	998 if (!scalar_replacement(alloc, safepoints)) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	999 return false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	1000 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	1001
1080 7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1002 CompileLog* log = C->log();
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1003 if (log != NULL) {
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1004 log->head("eliminate_allocation type='%d'",
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1005 log->identify(tklass->klass()));
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1006 JVMState* p = alloc->jvms();
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1007 while (p != NULL) {
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1008 log->elem("jvms bci='%d' method='%d'", p->bci(), log->identify(p->method()));
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1009 p = p->caller();
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1010 }
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1011 log->tail("eliminate_allocation");
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1012 }
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1013
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	1014 process_users_of_allocation(alloc);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	1015
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	1016 #ifndef PRODUCT
1080 7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1017 if (PrintEliminateAllocations) {
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1018 if (alloc->is_AllocateArray())
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1019 tty->print_cr("++++ Eliminated: %d AllocateArray", alloc->_idx);
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1020 else
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1021 tty->print_cr("++++ Eliminated: %d Allocate", alloc->_idx);
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1022 }
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	1023 #endif
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	1024
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	1025 return true;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	1026 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	1027
10278 6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1028 bool PhaseMacroExpand::eliminate_boxing_node(CallStaticJavaNode *boxing) {
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1029 // EA should remove all uses of non-escaping boxing node.
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1030 if (!C->eliminate_boxing() \|\| boxing->proj_out(TypeFunc::Parms) != NULL) {
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1031 return false;
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1032 }
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1033
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1034 extract_call_projections(boxing);
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1035
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1036 const TypeTuple* r = boxing->tf()->range();
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1037 assert(r->cnt() > TypeFunc::Parms, "sanity");
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1038 const TypeInstPtr* t = r->field_at(TypeFunc::Parms)->isa_instptr();
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1039 assert(t != NULL, "sanity");
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1040
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1041 CompileLog* log = C->log();
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1042 if (log != NULL) {
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1043 log->head("eliminate_boxing type='%d'",
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1044 log->identify(t->klass()));
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1045 JVMState* p = boxing->jvms();
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1046 while (p != NULL) {
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1047 log->elem("jvms bci='%d' method='%d'", p->bci(), log->identify(p->method()));
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1048 p = p->caller();
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1049 }
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1050 log->tail("eliminate_boxing");
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1051 }
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1052
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1053 process_users_of_allocation(boxing);
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1054
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1055 #ifndef PRODUCT
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1056 if (PrintEliminateAllocations) {
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1057 tty->print("++++ Eliminated: %d ", boxing->_idx);
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1058 boxing->method()->print_short_name(tty);
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1059 tty->cr();
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1060 }
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1061 #endif
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1062
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1063 return true;
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	1064 }
0 a61af66fc99e Initial load duke parents: diff changeset	1065
a61af66fc99e Initial load duke parents: diff changeset	1066 //---------------------------set_eden_pointers-------------------------
a61af66fc99e Initial load duke parents: diff changeset	1067 void PhaseMacroExpand::set_eden_pointers(Node* &eden_top_adr, Node* &eden_end_adr) {
a61af66fc99e Initial load duke parents: diff changeset	1068 if (UseTLAB) { // Private allocation: load from TLS
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1069 Node* thread = transform_later(new (C) ThreadLocalNode());
0 a61af66fc99e Initial load duke parents: diff changeset	1070 int tlab_top_offset = in_bytes(JavaThread::tlab_top_offset());
a61af66fc99e Initial load duke parents: diff changeset	1071 int tlab_end_offset = in_bytes(JavaThread::tlab_end_offset());
a61af66fc99e Initial load duke parents: diff changeset	1072 eden_top_adr = basic_plus_adr(top()/not oop/, thread, tlab_top_offset);
a61af66fc99e Initial load duke parents: diff changeset	1073 eden_end_adr = basic_plus_adr(top()/not oop/, thread, tlab_end_offset);
a61af66fc99e Initial load duke parents: diff changeset	1074 } else { // Shared allocation: load from globals
a61af66fc99e Initial load duke parents: diff changeset	1075 CollectedHeap* ch = Universe::heap();
a61af66fc99e Initial load duke parents: diff changeset	1076 address top_adr = (address)ch->top_addr();
a61af66fc99e Initial load duke parents: diff changeset	1077 address end_adr = (address)ch->end_addr();
a61af66fc99e Initial load duke parents: diff changeset	1078 eden_top_adr = makecon(TypeRawPtr::make(top_adr));
a61af66fc99e Initial load duke parents: diff changeset	1079 eden_end_adr = basic_plus_adr(eden_top_adr, end_adr - top_adr);
a61af66fc99e Initial load duke parents: diff changeset	1080 }
a61af66fc99e Initial load duke parents: diff changeset	1081 }
a61af66fc99e Initial load duke parents: diff changeset	1082
a61af66fc99e Initial load duke parents: diff changeset	1083
a61af66fc99e Initial load duke parents: diff changeset	1084 Node* PhaseMacroExpand::make_load(Node* ctl, Node* mem, Node* base, int offset, const Type* value_type, BasicType bt) {
a61af66fc99e Initial load duke parents: diff changeset	1085 Node* adr = basic_plus_adr(base, offset);
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1086 const TypePtr* adr_type = adr->bottom_type()->is_ptr();
14429 2113136690bc 8024921: PPC64 (part 113): Extend Load and Store nodes to know about memory ordering goetz parents: 12226 diff changeset	1087 Node* value = LoadNode::make(_igvn, ctl, mem, adr, adr_type, value_type, bt, MemNode::unordered);
0 a61af66fc99e Initial load duke parents: diff changeset	1088 transform_later(value);
a61af66fc99e Initial load duke parents: diff changeset	1089 return value;
a61af66fc99e Initial load duke parents: diff changeset	1090 }
a61af66fc99e Initial load duke parents: diff changeset	1091
a61af66fc99e Initial load duke parents: diff changeset	1092
a61af66fc99e Initial load duke parents: diff changeset	1093 Node* PhaseMacroExpand::make_store(Node* ctl, Node* mem, Node* base, int offset, Node* value, BasicType bt) {
a61af66fc99e Initial load duke parents: diff changeset	1094 Node* adr = basic_plus_adr(base, offset);
14429 2113136690bc 8024921: PPC64 (part 113): Extend Load and Store nodes to know about memory ordering goetz parents: 12226 diff changeset	1095 mem = StoreNode::make(_igvn, ctl, mem, adr, NULL, value, bt, MemNode::unordered);
0 a61af66fc99e Initial load duke parents: diff changeset	1096 transform_later(mem);
a61af66fc99e Initial load duke parents: diff changeset	1097 return mem;
a61af66fc99e Initial load duke parents: diff changeset	1098 }
a61af66fc99e Initial load duke parents: diff changeset	1099
a61af66fc99e Initial load duke parents: diff changeset	1100 //=============================================================================
a61af66fc99e Initial load duke parents: diff changeset	1101 //
a61af66fc99e Initial load duke parents: diff changeset	1102 // A L L O C A T I O N
a61af66fc99e Initial load duke parents: diff changeset	1103 //
a61af66fc99e Initial load duke parents: diff changeset	1104 // Allocation attempts to be fast in the case of frequent small objects.
a61af66fc99e Initial load duke parents: diff changeset	1105 // It breaks down like this:
a61af66fc99e Initial load duke parents: diff changeset	1106 //
a61af66fc99e Initial load duke parents: diff changeset	1107 // 1) Size in doublewords is computed. This is a constant for objects and
a61af66fc99e Initial load duke parents: diff changeset	1108 // variable for most arrays. Doubleword units are used to avoid size
a61af66fc99e Initial load duke parents: diff changeset	1109 // overflow of huge doubleword arrays. We need doublewords in the end for
a61af66fc99e Initial load duke parents: diff changeset	1110 // rounding.
a61af66fc99e Initial load duke parents: diff changeset	1111 //
a61af66fc99e Initial load duke parents: diff changeset	1112 // 2) Size is checked for being 'too large'. Too-large allocations will go
a61af66fc99e Initial load duke parents: diff changeset	1113 // the slow path into the VM. The slow path can throw any required
a61af66fc99e Initial load duke parents: diff changeset	1114 // exceptions, and does all the special checks for very large arrays. The
a61af66fc99e Initial load duke parents: diff changeset	1115 // size test can constant-fold away for objects. For objects with
a61af66fc99e Initial load duke parents: diff changeset	1116 // finalizers it constant-folds the otherway: you always go slow with
a61af66fc99e Initial load duke parents: diff changeset	1117 // finalizers.
a61af66fc99e Initial load duke parents: diff changeset	1118 //
a61af66fc99e Initial load duke parents: diff changeset	1119 // 3) If NOT using TLABs, this is the contended loop-back point.
a61af66fc99e Initial load duke parents: diff changeset	1120 // Load-Locked the heap top. If using TLABs normal-load the heap top.
a61af66fc99e Initial load duke parents: diff changeset	1121 //
a61af66fc99e Initial load duke parents: diff changeset	1122 // 4) Check that heap top + size*8 < max. If we fail go the slow ` route.
a61af66fc99e Initial load duke parents: diff changeset	1123 // NOTE: "top+size*8" cannot wrap the 4Gig line! Here's why: for largish
a61af66fc99e Initial load duke parents: diff changeset	1124 // "size*8" we always enter the VM, where "largish" is a constant picked small
a61af66fc99e Initial load duke parents: diff changeset	1125 // enough that there's always space between the eden max and 4Gig (old space is
a61af66fc99e Initial load duke parents: diff changeset	1126 // there so it's quite large) and large enough that the cost of entering the VM
a61af66fc99e Initial load duke parents: diff changeset	1127 // is dwarfed by the cost to initialize the space.
a61af66fc99e Initial load duke parents: diff changeset	1128 //
a61af66fc99e Initial load duke parents: diff changeset	1129 // 5) If NOT using TLABs, Store-Conditional the adjusted heap top back
a61af66fc99e Initial load duke parents: diff changeset	1130 // down. If contended, repeat at step 3. If using TLABs normal-store
a61af66fc99e Initial load duke parents: diff changeset	1131 // adjusted heap top back down; there is no contention.
a61af66fc99e Initial load duke parents: diff changeset	1132 //
a61af66fc99e Initial load duke parents: diff changeset	1133 // 6) If !ZeroTLAB then Bulk-clear the object/array. Fill in klass & mark
a61af66fc99e Initial load duke parents: diff changeset	1134 // fields.
a61af66fc99e Initial load duke parents: diff changeset	1135 //
a61af66fc99e Initial load duke parents: diff changeset	1136 // 7) Merge with the slow-path; cast the raw memory pointer to the correct
a61af66fc99e Initial load duke parents: diff changeset	1137 // oop flavor.
a61af66fc99e Initial load duke parents: diff changeset	1138 //
a61af66fc99e Initial load duke parents: diff changeset	1139 //=============================================================================
a61af66fc99e Initial load duke parents: diff changeset	1140 // FastAllocateSizeLimit value is in DOUBLEWORDS.
a61af66fc99e Initial load duke parents: diff changeset	1141 // Allocations bigger than this always go the slow route.
a61af66fc99e Initial load duke parents: diff changeset	1142 // This value must be small enough that allocation attempts that need to
a61af66fc99e Initial load duke parents: diff changeset	1143 // trigger exceptions go the slow route. Also, it must be small enough so
a61af66fc99e Initial load duke parents: diff changeset	1144 // that heap_top + size_in_bytes does not wrap around the 4Gig limit.
a61af66fc99e Initial load duke parents: diff changeset	1145 //=============================================================================j//
a61af66fc99e Initial load duke parents: diff changeset	1146 // %%% Here is an old comment from parseHelper.cpp; is it outdated?
a61af66fc99e Initial load duke parents: diff changeset	1147 // The allocator will coalesce int->oop copies away. See comment in
a61af66fc99e Initial load duke parents: diff changeset	1148 // coalesce.cpp about how this works. It depends critically on the exact
a61af66fc99e Initial load duke parents: diff changeset	1149 // code shape produced here, so if you are changing this code shape
a61af66fc99e Initial load duke parents: diff changeset	1150 // make sure the GC info for the heap-top is correct in and around the
a61af66fc99e Initial load duke parents: diff changeset	1151 // slow-path call.
a61af66fc99e Initial load duke parents: diff changeset	1152 //
a61af66fc99e Initial load duke parents: diff changeset	1153
a61af66fc99e Initial load duke parents: diff changeset	1154 void PhaseMacroExpand::expand_allocate_common(
a61af66fc99e Initial load duke parents: diff changeset	1155 AllocateNode* alloc, // allocation node to be expanded
a61af66fc99e Initial load duke parents: diff changeset	1156 Node* length, // array length for an array allocation
a61af66fc99e Initial load duke parents: diff changeset	1157 const TypeFunc* slow_call_type, // Type of slow call
a61af66fc99e Initial load duke parents: diff changeset	1158 address slow_call_address // Address of slow call
a61af66fc99e Initial load duke parents: diff changeset	1159 )
a61af66fc99e Initial load duke parents: diff changeset	1160 {
a61af66fc99e Initial load duke parents: diff changeset	1161
a61af66fc99e Initial load duke parents: diff changeset	1162 Node* ctrl = alloc->in(TypeFunc::Control);
a61af66fc99e Initial load duke parents: diff changeset	1163 Node* mem = alloc->in(TypeFunc::Memory);
a61af66fc99e Initial load duke parents: diff changeset	1164 Node* i_o = alloc->in(TypeFunc::I_O);
a61af66fc99e Initial load duke parents: diff changeset	1165 Node* size_in_bytes = alloc->in(AllocateNode::AllocSize);
a61af66fc99e Initial load duke parents: diff changeset	1166 Node* klass_node = alloc->in(AllocateNode::KlassNode);
a61af66fc99e Initial load duke parents: diff changeset	1167 Node* initial_slow_test = alloc->in(AllocateNode::InitialTest);
a61af66fc99e Initial load duke parents: diff changeset	1168
a61af66fc99e Initial load duke parents: diff changeset	1169 assert(ctrl != NULL, "must have control");
a61af66fc99e Initial load duke parents: diff changeset	1170 // We need a Region and corresponding Phi's to merge the slow-path and fast-path results.
a61af66fc99e Initial load duke parents: diff changeset	1171 // they will not be used if "always_slow" is set
a61af66fc99e Initial load duke parents: diff changeset	1172 enum { slow_result_path = 1, fast_result_path = 2 };
a61af66fc99e Initial load duke parents: diff changeset	1173 Node *result_region;
a61af66fc99e Initial load duke parents: diff changeset	1174 Node *result_phi_rawmem;
a61af66fc99e Initial load duke parents: diff changeset	1175 Node *result_phi_rawoop;
a61af66fc99e Initial load duke parents: diff changeset	1176 Node *result_phi_i_o;
a61af66fc99e Initial load duke parents: diff changeset	1177
a61af66fc99e Initial load duke parents: diff changeset	1178 // The initial slow comparison is a size check, the comparison
a61af66fc99e Initial load duke parents: diff changeset	1179 // we want to do is a BoolTest::gt
a61af66fc99e Initial load duke parents: diff changeset	1180 bool always_slow = false;
a61af66fc99e Initial load duke parents: diff changeset	1181 int tv = _igvn.find_int_con(initial_slow_test, -1);
a61af66fc99e Initial load duke parents: diff changeset	1182 if (tv >= 0) {
a61af66fc99e Initial load duke parents: diff changeset	1183 always_slow = (tv == 1);
a61af66fc99e Initial load duke parents: diff changeset	1184 initial_slow_test = NULL;
a61af66fc99e Initial load duke parents: diff changeset	1185 } else {
a61af66fc99e Initial load duke parents: diff changeset	1186 initial_slow_test = BoolNode::make_predicate(initial_slow_test, &_igvn);
a61af66fc99e Initial load duke parents: diff changeset	1187 }
a61af66fc99e Initial load duke parents: diff changeset	1188
780 c96bf21b756f 6788527: Server vm intermittently fails with assertion "live value must not be garbage" with fastdebug bits kvn parents: 708 diff changeset	1189 if (C->env()->dtrace_alloc_probes() \|\|
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1190 !UseTLAB && (!Universe::heap()->supports_inline_contig_alloc() \|\|
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1191 (UseConcMarkSweepGC && CMSIncrementalMode))) {
0 a61af66fc99e Initial load duke parents: diff changeset	1192 // Force slow-path allocation
a61af66fc99e Initial load duke parents: diff changeset	1193 always_slow = true;
a61af66fc99e Initial load duke parents: diff changeset	1194 initial_slow_test = NULL;
a61af66fc99e Initial load duke parents: diff changeset	1195 }
a61af66fc99e Initial load duke parents: diff changeset	1196
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1197
0 a61af66fc99e Initial load duke parents: diff changeset	1198 enum { too_big_or_final_path = 1, need_gc_path = 2 };
a61af66fc99e Initial load duke parents: diff changeset	1199 Node *slow_region = NULL;
a61af66fc99e Initial load duke parents: diff changeset	1200 Node *toobig_false = ctrl;
a61af66fc99e Initial load duke parents: diff changeset	1201
a61af66fc99e Initial load duke parents: diff changeset	1202 assert (initial_slow_test == NULL \|\| !always_slow, "arguments must be consistent");
a61af66fc99e Initial load duke parents: diff changeset	1203 // generate the initial test if necessary
a61af66fc99e Initial load duke parents: diff changeset	1204 if (initial_slow_test != NULL ) {
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1205 slow_region = new (C) RegionNode(3);
0 a61af66fc99e Initial load duke parents: diff changeset	1206
a61af66fc99e Initial load duke parents: diff changeset	1207 // Now make the initial failure test. Usually a too-big test but
a61af66fc99e Initial load duke parents: diff changeset	1208 // might be a TRUE for finalizers or a fancy class check for
a61af66fc99e Initial load duke parents: diff changeset	1209 // newInstance0.
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1210 IfNode *toobig_iff = new (C) IfNode(ctrl, initial_slow_test, PROB_MIN, COUNT_UNKNOWN);
0 a61af66fc99e Initial load duke parents: diff changeset	1211 transform_later(toobig_iff);
a61af66fc99e Initial load duke parents: diff changeset	1212 // Plug the failing-too-big test into the slow-path region
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1213 Node *toobig_true = new (C) IfTrueNode( toobig_iff );
0 a61af66fc99e Initial load duke parents: diff changeset	1214 transform_later(toobig_true);
a61af66fc99e Initial load duke parents: diff changeset	1215 slow_region ->init_req( too_big_or_final_path, toobig_true );
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1216 toobig_false = new (C) IfFalseNode( toobig_iff );
0 a61af66fc99e Initial load duke parents: diff changeset	1217 transform_later(toobig_false);
a61af66fc99e Initial load duke parents: diff changeset	1218 } else { // No initial test, just fall into next case
a61af66fc99e Initial load duke parents: diff changeset	1219 toobig_false = ctrl;
a61af66fc99e Initial load duke parents: diff changeset	1220 debug_only(slow_region = NodeSentinel);
a61af66fc99e Initial load duke parents: diff changeset	1221 }
a61af66fc99e Initial load duke parents: diff changeset	1222
a61af66fc99e Initial load duke parents: diff changeset	1223 Node *slow_mem = mem; // save the current memory state for slow path
a61af66fc99e Initial load duke parents: diff changeset	1224 // generate the fast allocation code unless we know that the initial test will always go slow
a61af66fc99e Initial load duke parents: diff changeset	1225 if (!always_slow) {
565 7fe62bb75bf4 6799693: Server compiler leads to data corruption when expression throws an Exception kvn parents: 490 diff changeset	1226 // Fast path modifies only raw memory.
7fe62bb75bf4 6799693: Server compiler leads to data corruption when expression throws an Exception kvn parents: 490 diff changeset	1227 if (mem->is_MergeMem()) {
7fe62bb75bf4 6799693: Server compiler leads to data corruption when expression throws an Exception kvn parents: 490 diff changeset	1228 mem = mem->as_MergeMem()->memory_at(Compile::AliasIdxRaw);
7fe62bb75bf4 6799693: Server compiler leads to data corruption when expression throws an Exception kvn parents: 490 diff changeset	1229 }
7fe62bb75bf4 6799693: Server compiler leads to data corruption when expression throws an Exception kvn parents: 490 diff changeset	1230
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1231 Node* eden_top_adr;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1232 Node* eden_end_adr;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1233
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1234 set_eden_pointers(eden_top_adr, eden_end_adr);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1235
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1236 // Load Eden::end. Loop invariant and hoisted.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1237 //
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1238 // Note: We set the control input on "eden_end" and "old_eden_top" when using
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1239 // a TLAB to work around a bug where these values were being moved across
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1240 // a safepoint. These are not oops, so they cannot be include in the oop
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1241 // map, but they can be changed by a GC. The proper way to fix this would
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1242 // be to set the raw memory state when generating a SafepointNode. However
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1243 // this will require extensive changes to the loop optimization in order to
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1244 // prevent a degradation of the optimization.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1245 // See comment in memnode.hpp, around line 227 in class LoadPNode.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1246 Node *eden_end = make_load(ctrl, mem, eden_end_adr, 0, TypeRawPtr::BOTTOM, T_ADDRESS);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1247
0 a61af66fc99e Initial load duke parents: diff changeset	1248 // allocate the Region and Phi nodes for the result
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1249 result_region = new (C) RegionNode(3);
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1250 result_phi_rawmem = new (C) PhiNode(result_region, Type::MEMORY, TypeRawPtr::BOTTOM);
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1251 result_phi_rawoop = new (C) PhiNode(result_region, TypeRawPtr::BOTTOM);
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1252 result_phi_i_o = new (C) PhiNode(result_region, Type::ABIO); // I/O is used for Prefetch
0 a61af66fc99e Initial load duke parents: diff changeset	1253
a61af66fc99e Initial load duke parents: diff changeset	1254 // We need a Region for the loop-back contended case.
a61af66fc99e Initial load duke parents: diff changeset	1255 enum { fall_in_path = 1, contended_loopback_path = 2 };
a61af66fc99e Initial load duke parents: diff changeset	1256 Node *contended_region;
a61af66fc99e Initial load duke parents: diff changeset	1257 Node *contended_phi_rawmem;
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1258 if (UseTLAB) {
0 a61af66fc99e Initial load duke parents: diff changeset	1259 contended_region = toobig_false;
a61af66fc99e Initial load duke parents: diff changeset	1260 contended_phi_rawmem = mem;
a61af66fc99e Initial load duke parents: diff changeset	1261 } else {
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1262 contended_region = new (C) RegionNode(3);
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1263 contended_phi_rawmem = new (C) PhiNode(contended_region, Type::MEMORY, TypeRawPtr::BOTTOM);
0 a61af66fc99e Initial load duke parents: diff changeset	1264 // Now handle the passing-too-big test. We fall into the contended
a61af66fc99e Initial load duke parents: diff changeset	1265 // loop-back merge point.
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1266 contended_region ->init_req(fall_in_path, toobig_false);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1267 contended_phi_rawmem->init_req(fall_in_path, mem);
0 a61af66fc99e Initial load duke parents: diff changeset	1268 transform_later(contended_region);
a61af66fc99e Initial load duke parents: diff changeset	1269 transform_later(contended_phi_rawmem);
a61af66fc99e Initial load duke parents: diff changeset	1270 }
a61af66fc99e Initial load duke parents: diff changeset	1271
a61af66fc99e Initial load duke parents: diff changeset	1272 // Load(-locked) the heap top.
a61af66fc99e Initial load duke parents: diff changeset	1273 // See note above concerning the control input when using a TLAB
a61af66fc99e Initial load duke parents: diff changeset	1274 Node *old_eden_top = UseTLAB
14429 2113136690bc 8024921: PPC64 (part 113): Extend Load and Store nodes to know about memory ordering goetz parents: 12226 diff changeset	1275 ? new (C) LoadPNode (ctrl, contended_phi_rawmem, eden_top_adr, TypeRawPtr::BOTTOM, TypeRawPtr::BOTTOM, MemNode::unordered)
2113136690bc 8024921: PPC64 (part 113): Extend Load and Store nodes to know about memory ordering goetz parents: 12226 diff changeset	1276 : new (C) LoadPLockedNode(contended_region, contended_phi_rawmem, eden_top_adr, MemNode::acquire);
0 a61af66fc99e Initial load duke parents: diff changeset	1277
a61af66fc99e Initial load duke parents: diff changeset	1278 transform_later(old_eden_top);
a61af66fc99e Initial load duke parents: diff changeset	1279 // Add to heap top to get a new heap top
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1280 Node *new_eden_top = new (C) AddPNode(top(), old_eden_top, size_in_bytes);
0 a61af66fc99e Initial load duke parents: diff changeset	1281 transform_later(new_eden_top);
a61af66fc99e Initial load duke parents: diff changeset	1282 // Check for needing a GC; compare against heap end
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1283 Node *needgc_cmp = new (C) CmpPNode(new_eden_top, eden_end);
0 a61af66fc99e Initial load duke parents: diff changeset	1284 transform_later(needgc_cmp);
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1285 Node *needgc_bol = new (C) BoolNode(needgc_cmp, BoolTest::ge);
0 a61af66fc99e Initial load duke parents: diff changeset	1286 transform_later(needgc_bol);
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1287 IfNode *needgc_iff = new (C) IfNode(contended_region, needgc_bol, PROB_UNLIKELY_MAG(4), COUNT_UNKNOWN);
0 a61af66fc99e Initial load duke parents: diff changeset	1288 transform_later(needgc_iff);
a61af66fc99e Initial load duke parents: diff changeset	1289
a61af66fc99e Initial load duke parents: diff changeset	1290 // Plug the failing-heap-space-need-gc test into the slow-path region
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1291 Node *needgc_true = new (C) IfTrueNode(needgc_iff);
0 a61af66fc99e Initial load duke parents: diff changeset	1292 transform_later(needgc_true);
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1293 if (initial_slow_test) {
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1294 slow_region->init_req(need_gc_path, needgc_true);
0 a61af66fc99e Initial load duke parents: diff changeset	1295 // This completes all paths into the slow merge point
a61af66fc99e Initial load duke parents: diff changeset	1296 transform_later(slow_region);
a61af66fc99e Initial load duke parents: diff changeset	1297 } else { // No initial slow path needed!
a61af66fc99e Initial load duke parents: diff changeset	1298 // Just fall from the need-GC path straight into the VM call.
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1299 slow_region = needgc_true;
0 a61af66fc99e Initial load duke parents: diff changeset	1300 }
a61af66fc99e Initial load duke parents: diff changeset	1301 // No need for a GC. Setup for the Store-Conditional
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1302 Node *needgc_false = new (C) IfFalseNode(needgc_iff);
0 a61af66fc99e Initial load duke parents: diff changeset	1303 transform_later(needgc_false);
a61af66fc99e Initial load duke parents: diff changeset	1304
a61af66fc99e Initial load duke parents: diff changeset	1305 // Grab regular I/O before optional prefetch may change it.
a61af66fc99e Initial load duke parents: diff changeset	1306 // Slow-path does no I/O so just set it to the original I/O.
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1307 result_phi_i_o->init_req(slow_result_path, i_o);
0 a61af66fc99e Initial load duke parents: diff changeset	1308
a61af66fc99e Initial load duke parents: diff changeset	1309 i_o = prefetch_allocation(i_o, needgc_false, contended_phi_rawmem,
a61af66fc99e Initial load duke parents: diff changeset	1310 old_eden_top, new_eden_top, length);
a61af66fc99e Initial load duke parents: diff changeset	1311
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1312 // Name successful fast-path variables
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1313 Node* fast_oop = old_eden_top;
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1314 Node* fast_oop_ctrl;
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1315 Node* fast_oop_rawmem;
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1316
0 a61af66fc99e Initial load duke parents: diff changeset	1317 // Store (-conditional) the modified eden top back down.
a61af66fc99e Initial load duke parents: diff changeset	1318 // StorePConditional produces flags for a test PLUS a modified raw
a61af66fc99e Initial load duke parents: diff changeset	1319 // memory state.
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1320 if (UseTLAB) {
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1321 Node* store_eden_top =
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1322 new (C) StorePNode(needgc_false, contended_phi_rawmem, eden_top_adr,
14429 2113136690bc 8024921: PPC64 (part 113): Extend Load and Store nodes to know about memory ordering goetz parents: 12226 diff changeset	1323 TypeRawPtr::BOTTOM, new_eden_top, MemNode::unordered);
0 a61af66fc99e Initial load duke parents: diff changeset	1324 transform_later(store_eden_top);
a61af66fc99e Initial load duke parents: diff changeset	1325 fast_oop_ctrl = needgc_false; // No contention, so this is the fast path
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1326 fast_oop_rawmem = store_eden_top;
0 a61af66fc99e Initial load duke parents: diff changeset	1327 } else {
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1328 Node* store_eden_top =
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1329 new (C) StorePConditionalNode(needgc_false, contended_phi_rawmem, eden_top_adr,
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1330 new_eden_top, fast_oop/old_eden_top/);
0 a61af66fc99e Initial load duke parents: diff changeset	1331 transform_later(store_eden_top);
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1332 Node *contention_check = new (C) BoolNode(store_eden_top, BoolTest::ne);
0 a61af66fc99e Initial load duke parents: diff changeset	1333 transform_later(contention_check);
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1334 store_eden_top = new (C) SCMemProjNode(store_eden_top);
0 a61af66fc99e Initial load duke parents: diff changeset	1335 transform_later(store_eden_top);
a61af66fc99e Initial load duke parents: diff changeset	1336
a61af66fc99e Initial load duke parents: diff changeset	1337 // If not using TLABs, check to see if there was contention.
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1338 IfNode *contention_iff = new (C) IfNode (needgc_false, contention_check, PROB_MIN, COUNT_UNKNOWN);
0 a61af66fc99e Initial load duke parents: diff changeset	1339 transform_later(contention_iff);
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1340 Node *contention_true = new (C) IfTrueNode(contention_iff);
0 a61af66fc99e Initial load duke parents: diff changeset	1341 transform_later(contention_true);
a61af66fc99e Initial load duke parents: diff changeset	1342 // If contention, loopback and try again.
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1343 contended_region->init_req(contended_loopback_path, contention_true);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1344 contended_phi_rawmem->init_req(contended_loopback_path, store_eden_top);
0 a61af66fc99e Initial load duke parents: diff changeset	1345
a61af66fc99e Initial load duke parents: diff changeset	1346 // Fast-path succeeded with no contention!
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1347 Node *contention_false = new (C) IfFalseNode(contention_iff);
0 a61af66fc99e Initial load duke parents: diff changeset	1348 transform_later(contention_false);
a61af66fc99e Initial load duke parents: diff changeset	1349 fast_oop_ctrl = contention_false;
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1350
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1351 // Bump total allocated bytes for this thread
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1352 Node* thread = new (C) ThreadLocalNode();
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1353 transform_later(thread);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1354 Node* alloc_bytes_adr = basic_plus_adr(top()/not oop/, thread,
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1355 in_bytes(JavaThread::allocated_bytes_offset()));
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1356 Node* alloc_bytes = make_load(fast_oop_ctrl, store_eden_top, alloc_bytes_adr,
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1357 0, TypeLong::LONG, T_LONG);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1358 #ifdef _LP64
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1359 Node* alloc_size = size_in_bytes;
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1360 #else
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1361 Node* alloc_size = new (C) ConvI2LNode(size_in_bytes);
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1362 transform_later(alloc_size);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1363 #endif
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1364 Node* new_alloc_bytes = new (C) AddLNode(alloc_bytes, alloc_size);
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1365 transform_later(new_alloc_bytes);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1366 fast_oop_rawmem = make_store(fast_oop_ctrl, store_eden_top, alloc_bytes_adr,
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1367 0, new_alloc_bytes, T_LONG);
0 a61af66fc99e Initial load duke parents: diff changeset	1368 }
a61af66fc99e Initial load duke parents: diff changeset	1369
4763 1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1370 InitializeNode* init = alloc->initialization();
0 a61af66fc99e Initial load duke parents: diff changeset	1371 fast_oop_rawmem = initialize_object(alloc,
a61af66fc99e Initial load duke parents: diff changeset	1372 fast_oop_ctrl, fast_oop_rawmem, fast_oop,
a61af66fc99e Initial load duke parents: diff changeset	1373 klass_node, length, size_in_bytes);
a61af66fc99e Initial load duke parents: diff changeset	1374
4763 1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1375 // If initialization is performed by an array copy, any required
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1376 // MemBarStoreStore was already added. If the object does not
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1377 // escape no need for a MemBarStoreStore. Otherwise we need a
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1378 // MemBarStoreStore so that stores that initialize this object
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1379 // can't be reordered with a subsequent store that makes this
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1380 // object accessible by other threads.
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1381 if (init == NULL \|\| (!init->is_complete_with_arraycopy() && !init->does_not_escape())) {
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1382 if (init == NULL \|\| init->req() < InitializeNode::RawStores) {
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1383 // No InitializeNode or no stores captured by zeroing
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1384 // elimination. Simply add the MemBarStoreStore after object
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1385 // initialization.
8694 8651f608fea4 8009460: C2compiler crash in machnode::in_regmask(unsigned int) roland parents: 7637 diff changeset	1386 MemBarNode* mb = MemBarNode::make(C, Op_MemBarStoreStore, Compile::AliasIdxBot);
4763 1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1387 transform_later(mb);
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1388
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1389 mb->init_req(TypeFunc::Memory, fast_oop_rawmem);
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1390 mb->init_req(TypeFunc::Control, fast_oop_ctrl);
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1391 fast_oop_ctrl = new (C) ProjNode(mb,TypeFunc::Control);
4763 1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1392 transform_later(fast_oop_ctrl);
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1393 fast_oop_rawmem = new (C) ProjNode(mb,TypeFunc::Memory);
4763 1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1394 transform_later(fast_oop_rawmem);
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1395 } else {
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1396 // Add the MemBarStoreStore after the InitializeNode so that
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1397 // all stores performing the initialization that were moved
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1398 // before the InitializeNode happen before the storestore
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1399 // barrier.
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1400
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1401 Node* init_ctrl = init->proj_out(TypeFunc::Control);
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1402 Node* init_mem = init->proj_out(TypeFunc::Memory);
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1403
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1404 MemBarNode* mb = MemBarNode::make(C, Op_MemBarStoreStore, Compile::AliasIdxBot);
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1405 transform_later(mb);
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1406
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1407 Node* ctrl = new (C) ProjNode(init,TypeFunc::Control);
4763 1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1408 transform_later(ctrl);
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1409 Node* mem = new (C) ProjNode(init,TypeFunc::Memory);
4763 1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1410 transform_later(mem);
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1411
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1412 // The MemBarStoreStore depends on control and memory coming
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1413 // from the InitializeNode
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1414 mb->init_req(TypeFunc::Memory, mem);
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1415 mb->init_req(TypeFunc::Control, ctrl);
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1416
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1417 ctrl = new (C) ProjNode(mb,TypeFunc::Control);
4763 1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1418 transform_later(ctrl);
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1419 mem = new (C) ProjNode(mb,TypeFunc::Memory);
4763 1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1420 transform_later(mem);
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1421
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1422 // All nodes that depended on the InitializeNode for control
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1423 // and memory must now depend on the MemBarNode that itself
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1424 // depends on the InitializeNode
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1425 _igvn.replace_node(init_ctrl, ctrl);
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1426 _igvn.replace_node(init_mem, mem);
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1427 }
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1428 }
1dc233a8c7fe 7121140: Allocation paths require explicit memory synchronization operations for RMO systems roland parents: 4762 diff changeset	1429
780 c96bf21b756f 6788527: Server vm intermittently fails with assertion "live value must not be garbage" with fastdebug bits kvn parents: 708 diff changeset	1430 if (C->env()->dtrace_extended_probes()) {
0 a61af66fc99e Initial load duke parents: diff changeset	1431 // Slow-path call
a61af66fc99e Initial load duke parents: diff changeset	1432 int size = TypeFunc::Parms + 2;
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1433 CallLeafNode *call = new (C) CallLeafNode(OptoRuntime::dtrace_object_alloc_Type(),
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1434 CAST_FROM_FN_PTR(address, SharedRuntime::dtrace_object_alloc_base),
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1435 "dtrace_object_alloc",
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1436 TypeRawPtr::BOTTOM);
0 a61af66fc99e Initial load duke parents: diff changeset	1437
a61af66fc99e Initial load duke parents: diff changeset	1438 // Get base of thread-local storage area
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1439 Node* thread = new (C) ThreadLocalNode();
0 a61af66fc99e Initial load duke parents: diff changeset	1440 transform_later(thread);
a61af66fc99e Initial load duke parents: diff changeset	1441
a61af66fc99e Initial load duke parents: diff changeset	1442 call->init_req(TypeFunc::Parms+0, thread);
a61af66fc99e Initial load duke parents: diff changeset	1443 call->init_req(TypeFunc::Parms+1, fast_oop);
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1444 call->init_req(TypeFunc::Control, fast_oop_ctrl);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1445 call->init_req(TypeFunc::I_O , top()); // does no i/o
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1446 call->init_req(TypeFunc::Memory , fast_oop_rawmem);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1447 call->init_req(TypeFunc::ReturnAdr, alloc->in(TypeFunc::ReturnAdr));
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1448 call->init_req(TypeFunc::FramePtr, alloc->in(TypeFunc::FramePtr));
0 a61af66fc99e Initial load duke parents: diff changeset	1449 transform_later(call);
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1450 fast_oop_ctrl = new (C) ProjNode(call,TypeFunc::Control);
0 a61af66fc99e Initial load duke parents: diff changeset	1451 transform_later(fast_oop_ctrl);
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1452 fast_oop_rawmem = new (C) ProjNode(call,TypeFunc::Memory);
0 a61af66fc99e Initial load duke parents: diff changeset	1453 transform_later(fast_oop_rawmem);
a61af66fc99e Initial load duke parents: diff changeset	1454 }
a61af66fc99e Initial load duke parents: diff changeset	1455
a61af66fc99e Initial load duke parents: diff changeset	1456 // Plug in the successful fast-path into the result merge point
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1457 result_region ->init_req(fast_result_path, fast_oop_ctrl);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1458 result_phi_rawoop->init_req(fast_result_path, fast_oop);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1459 result_phi_i_o ->init_req(fast_result_path, i_o);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1460 result_phi_rawmem->init_req(fast_result_path, fast_oop_rawmem);
0 a61af66fc99e Initial load duke parents: diff changeset	1461 } else {
a61af66fc99e Initial load duke parents: diff changeset	1462 slow_region = ctrl;
4767 d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1463 result_phi_i_o = i_o; // Rename it to use in the following code.
0 a61af66fc99e Initial load duke parents: diff changeset	1464 }
a61af66fc99e Initial load duke parents: diff changeset	1465
a61af66fc99e Initial load duke parents: diff changeset	1466 // Generate slow-path call
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1467 CallNode *call = new (C) CallStaticJavaNode(slow_call_type, slow_call_address,
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1468 OptoRuntime::stub_name(slow_call_address),
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1469 alloc->jvms()->bci(),
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1470 TypePtr::BOTTOM);
0 a61af66fc99e Initial load duke parents: diff changeset	1471 call->init_req( TypeFunc::Control, slow_region );
a61af66fc99e Initial load duke parents: diff changeset	1472 call->init_req( TypeFunc::I_O , top() ) ; // does no i/o
a61af66fc99e Initial load duke parents: diff changeset	1473 call->init_req( TypeFunc::Memory , slow_mem ); // may gc ptrs
a61af66fc99e Initial load duke parents: diff changeset	1474 call->init_req( TypeFunc::ReturnAdr, alloc->in(TypeFunc::ReturnAdr) );
a61af66fc99e Initial load duke parents: diff changeset	1475 call->init_req( TypeFunc::FramePtr, alloc->in(TypeFunc::FramePtr) );
a61af66fc99e Initial load duke parents: diff changeset	1476
a61af66fc99e Initial load duke parents: diff changeset	1477 call->init_req(TypeFunc::Parms+0, klass_node);
a61af66fc99e Initial load duke parents: diff changeset	1478 if (length != NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1479 call->init_req(TypeFunc::Parms+1, length);
a61af66fc99e Initial load duke parents: diff changeset	1480 }
a61af66fc99e Initial load duke parents: diff changeset	1481
a61af66fc99e Initial load duke parents: diff changeset	1482 // Copy debug information and adjust JVMState information, then replace
a61af66fc99e Initial load duke parents: diff changeset	1483 // allocate node with the call
a61af66fc99e Initial load duke parents: diff changeset	1484 copy_call_debug_info((CallNode *) alloc, call);
a61af66fc99e Initial load duke parents: diff changeset	1485 if (!always_slow) {
a61af66fc99e Initial load duke parents: diff changeset	1486 call->set_cnt(PROB_UNLIKELY_MAG(4)); // Same effect as RC_UNCOMMON.
4767 d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1487 } else {
d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1488 // Hook i_o projection to avoid its elimination during allocation
d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1489 // replacement (when only a slow call is generated).
d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1490 call->set_req(TypeFunc::I_O, result_phi_i_o);
0 a61af66fc99e Initial load duke parents: diff changeset	1491 }
1621 6027dddc26c6 6677629: PhaseIterGVN::subsume_node() should call hash_delete() and add_users_to_worklist() kvn parents: 1609 diff changeset	1492 _igvn.replace_node(alloc, call);
0 a61af66fc99e Initial load duke parents: diff changeset	1493 transform_later(call);
a61af66fc99e Initial load duke parents: diff changeset	1494
a61af66fc99e Initial load duke parents: diff changeset	1495 // Identify the output projections from the allocate node and
a61af66fc99e Initial load duke parents: diff changeset	1496 // adjust any references to them.
a61af66fc99e Initial load duke parents: diff changeset	1497 // The control and io projections look like:
a61af66fc99e Initial load duke parents: diff changeset	1498 //
a61af66fc99e Initial load duke parents: diff changeset	1499 // v---Proj(ctrl) <-----+ v---CatchProj(ctrl)
a61af66fc99e Initial load duke parents: diff changeset	1500 // Allocate Catch
a61af66fc99e Initial load duke parents: diff changeset	1501 // ^---Proj(io) <-------+ ^---CatchProj(io)
a61af66fc99e Initial load duke parents: diff changeset	1502 //
a61af66fc99e Initial load duke parents: diff changeset	1503 // We are interested in the CatchProj nodes.
a61af66fc99e Initial load duke parents: diff changeset	1504 //
a61af66fc99e Initial load duke parents: diff changeset	1505 extract_call_projections(call);
a61af66fc99e Initial load duke parents: diff changeset	1506
4767 d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1507 // An allocate node has separate memory projections for the uses on
d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1508 // the control and i_o paths. Replace the control memory projection with
d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1509 // result_phi_rawmem (unless we are only generating a slow call when
d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1510 // both memory projections are combined)
0 a61af66fc99e Initial load duke parents: diff changeset	1511 if (!always_slow && _memproj_fallthrough != NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1512 for (DUIterator_Fast imax, i = _memproj_fallthrough->fast_outs(imax); i < imax; i++) {
a61af66fc99e Initial load duke parents: diff changeset	1513 Node *use = _memproj_fallthrough->fast_out(i);
6144 5e990493719e 7173340: C2: code cleanup: use PhaseIterGVN::replace_edge(Node, int, Node) where applicable kvn parents: 4894 diff changeset	1514 _igvn.rehash_node_delayed(use);
0 a61af66fc99e Initial load duke parents: diff changeset	1515 imax -= replace_input(use, _memproj_fallthrough, result_phi_rawmem);
a61af66fc99e Initial load duke parents: diff changeset	1516 // back up iterator
a61af66fc99e Initial load duke parents: diff changeset	1517 --i;
a61af66fc99e Initial load duke parents: diff changeset	1518 }
a61af66fc99e Initial load duke parents: diff changeset	1519 }
4767 d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1520 // Now change uses of _memproj_catchall to use _memproj_fallthrough and delete
d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1521 // _memproj_catchall so we end up with a call that has only 1 memory projection.
0 a61af66fc99e Initial load duke parents: diff changeset	1522 if (_memproj_catchall != NULL ) {
a61af66fc99e Initial load duke parents: diff changeset	1523 if (_memproj_fallthrough == NULL) {
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1524 _memproj_fallthrough = new (C) ProjNode(call, TypeFunc::Memory);
0 a61af66fc99e Initial load duke parents: diff changeset	1525 transform_later(_memproj_fallthrough);
a61af66fc99e Initial load duke parents: diff changeset	1526 }
a61af66fc99e Initial load duke parents: diff changeset	1527 for (DUIterator_Fast imax, i = _memproj_catchall->fast_outs(imax); i < imax; i++) {
a61af66fc99e Initial load duke parents: diff changeset	1528 Node *use = _memproj_catchall->fast_out(i);
6144 5e990493719e 7173340: C2: code cleanup: use PhaseIterGVN::replace_edge(Node, int, Node) where applicable kvn parents: 4894 diff changeset	1529 _igvn.rehash_node_delayed(use);
0 a61af66fc99e Initial load duke parents: diff changeset	1530 imax -= replace_input(use, _memproj_catchall, _memproj_fallthrough);
a61af66fc99e Initial load duke parents: diff changeset	1531 // back up iterator
a61af66fc99e Initial load duke parents: diff changeset	1532 --i;
a61af66fc99e Initial load duke parents: diff changeset	1533 }
4767 d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1534 assert(_memproj_catchall->outcnt() == 0, "all uses must be deleted");
d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1535 _igvn.remove_dead_node(_memproj_catchall);
0 a61af66fc99e Initial load duke parents: diff changeset	1536 }
a61af66fc99e Initial load duke parents: diff changeset	1537
4767 d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1538 // An allocate node has separate i_o projections for the uses on the control
d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1539 // and i_o paths. Always replace the control i_o projection with result i_o
d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1540 // otherwise incoming i_o become dead when only a slow call is generated
d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1541 // (it is different from memory projections where both projections are
d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1542 // combined in such case).
d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1543 if (_ioproj_fallthrough != NULL) {
0 a61af66fc99e Initial load duke parents: diff changeset	1544 for (DUIterator_Fast imax, i = _ioproj_fallthrough->fast_outs(imax); i < imax; i++) {
a61af66fc99e Initial load duke parents: diff changeset	1545 Node *use = _ioproj_fallthrough->fast_out(i);
6144 5e990493719e 7173340: C2: code cleanup: use PhaseIterGVN::replace_edge(Node, int, Node) where applicable kvn parents: 4894 diff changeset	1546 _igvn.rehash_node_delayed(use);
0 a61af66fc99e Initial load duke parents: diff changeset	1547 imax -= replace_input(use, _ioproj_fallthrough, result_phi_i_o);
a61af66fc99e Initial load duke parents: diff changeset	1548 // back up iterator
a61af66fc99e Initial load duke parents: diff changeset	1549 --i;
a61af66fc99e Initial load duke parents: diff changeset	1550 }
a61af66fc99e Initial load duke parents: diff changeset	1551 }
4767 d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1552 // Now change uses of _ioproj_catchall to use _ioproj_fallthrough and delete
d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1553 // _ioproj_catchall so we end up with a call that has only 1 i_o projection.
0 a61af66fc99e Initial load duke parents: diff changeset	1554 if (_ioproj_catchall != NULL ) {
4767 d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1555 if (_ioproj_fallthrough == NULL) {
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1556 _ioproj_fallthrough = new (C) ProjNode(call, TypeFunc::I_O);
4767 d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1557 transform_later(_ioproj_fallthrough);
d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1558 }
0 a61af66fc99e Initial load duke parents: diff changeset	1559 for (DUIterator_Fast imax, i = _ioproj_catchall->fast_outs(imax); i < imax; i++) {
a61af66fc99e Initial load duke parents: diff changeset	1560 Node *use = _ioproj_catchall->fast_out(i);
6144 5e990493719e 7173340: C2: code cleanup: use PhaseIterGVN::replace_edge(Node, int, Node) where applicable kvn parents: 4894 diff changeset	1561 _igvn.rehash_node_delayed(use);
0 a61af66fc99e Initial load duke parents: diff changeset	1562 imax -= replace_input(use, _ioproj_catchall, _ioproj_fallthrough);
a61af66fc99e Initial load duke parents: diff changeset	1563 // back up iterator
a61af66fc99e Initial load duke parents: diff changeset	1564 --i;
a61af66fc99e Initial load duke parents: diff changeset	1565 }
4767 d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1566 assert(_ioproj_catchall->outcnt() == 0, "all uses must be deleted");
d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1567 _igvn.remove_dead_node(_ioproj_catchall);
0 a61af66fc99e Initial load duke parents: diff changeset	1568 }
a61af66fc99e Initial load duke parents: diff changeset	1569
a61af66fc99e Initial load duke parents: diff changeset	1570 // if we generated only a slow call, we are done
4767 d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1571 if (always_slow) {
d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1572 // Now we can unhook i_o.
4769 9c87bcb3b4dd 7125879: assert(proj != NULL) failed: must be found kvn parents: 4767 diff changeset	1573 if (result_phi_i_o->outcnt() > 1) {
9c87bcb3b4dd 7125879: assert(proj != NULL) failed: must be found kvn parents: 4767 diff changeset	1574 call->set_req(TypeFunc::I_O, top());
9c87bcb3b4dd 7125879: assert(proj != NULL) failed: must be found kvn parents: 4767 diff changeset	1575 } else {
9c87bcb3b4dd 7125879: assert(proj != NULL) failed: must be found kvn parents: 4767 diff changeset	1576 assert(result_phi_i_o->unique_ctrl_out() == call, "");
9c87bcb3b4dd 7125879: assert(proj != NULL) failed: must be found kvn parents: 4767 diff changeset	1577 // Case of new array with negative size known during compilation.
9c87bcb3b4dd 7125879: assert(proj != NULL) failed: must be found kvn parents: 4767 diff changeset	1578 // AllocateArrayNode::Ideal() optimization disconnect unreachable
9c87bcb3b4dd 7125879: assert(proj != NULL) failed: must be found kvn parents: 4767 diff changeset	1579 // following code since call to runtime will throw exception.
9c87bcb3b4dd 7125879: assert(proj != NULL) failed: must be found kvn parents: 4767 diff changeset	1580 // As result there will be no users of i_o after the call.
9c87bcb3b4dd 7125879: assert(proj != NULL) failed: must be found kvn parents: 4767 diff changeset	1581 // Leave i_o attached to this call to avoid problems in preceding graph.
9c87bcb3b4dd 7125879: assert(proj != NULL) failed: must be found kvn parents: 4767 diff changeset	1582 }
0 a61af66fc99e Initial load duke parents: diff changeset	1583 return;
4767 d12a66fa3820 7123954: Some CTW test crash with SIGSEGV kvn parents: 4763 diff changeset	1584 }
0 a61af66fc99e Initial load duke parents: diff changeset	1585
a61af66fc99e Initial load duke parents: diff changeset	1586
a61af66fc99e Initial load duke parents: diff changeset	1587 if (_fallthroughcatchproj != NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1588 ctrl = _fallthroughcatchproj->clone();
a61af66fc99e Initial load duke parents: diff changeset	1589 transform_later(ctrl);
708 f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	1590 _igvn.replace_node(_fallthroughcatchproj, result_region);
0 a61af66fc99e Initial load duke parents: diff changeset	1591 } else {
a61af66fc99e Initial load duke parents: diff changeset	1592 ctrl = top();
a61af66fc99e Initial load duke parents: diff changeset	1593 }
a61af66fc99e Initial load duke parents: diff changeset	1594 Node *slow_result;
a61af66fc99e Initial load duke parents: diff changeset	1595 if (_resproj == NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1596 // no uses of the allocation result
a61af66fc99e Initial load duke parents: diff changeset	1597 slow_result = top();
a61af66fc99e Initial load duke parents: diff changeset	1598 } else {
a61af66fc99e Initial load duke parents: diff changeset	1599 slow_result = _resproj->clone();
a61af66fc99e Initial load duke parents: diff changeset	1600 transform_later(slow_result);
708 f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	1601 _igvn.replace_node(_resproj, result_phi_rawoop);
0 a61af66fc99e Initial load duke parents: diff changeset	1602 }
a61af66fc99e Initial load duke parents: diff changeset	1603
a61af66fc99e Initial load duke parents: diff changeset	1604 // Plug slow-path into result merge point
a61af66fc99e Initial load duke parents: diff changeset	1605 result_region ->init_req( slow_result_path, ctrl );
a61af66fc99e Initial load duke parents: diff changeset	1606 result_phi_rawoop->init_req( slow_result_path, slow_result);
a61af66fc99e Initial load duke parents: diff changeset	1607 result_phi_rawmem->init_req( slow_result_path, _memproj_fallthrough );
a61af66fc99e Initial load duke parents: diff changeset	1608 transform_later(result_region);
a61af66fc99e Initial load duke parents: diff changeset	1609 transform_later(result_phi_rawoop);
a61af66fc99e Initial load duke parents: diff changeset	1610 transform_later(result_phi_rawmem);
a61af66fc99e Initial load duke parents: diff changeset	1611 transform_later(result_phi_i_o);
a61af66fc99e Initial load duke parents: diff changeset	1612 // This completes all paths into the result merge point
a61af66fc99e Initial load duke parents: diff changeset	1613 }
a61af66fc99e Initial load duke parents: diff changeset	1614
a61af66fc99e Initial load duke parents: diff changeset	1615
a61af66fc99e Initial load duke parents: diff changeset	1616 // Helper for PhaseMacroExpand::expand_allocate_common.
a61af66fc99e Initial load duke parents: diff changeset	1617 // Initializes the newly-allocated storage.
a61af66fc99e Initial load duke parents: diff changeset	1618 Node*
a61af66fc99e Initial load duke parents: diff changeset	1619 PhaseMacroExpand::initialize_object(AllocateNode* alloc,
a61af66fc99e Initial load duke parents: diff changeset	1620 Node* control, Node* rawmem, Node* object,
a61af66fc99e Initial load duke parents: diff changeset	1621 Node* klass_node, Node* length,
a61af66fc99e Initial load duke parents: diff changeset	1622 Node* size_in_bytes) {
a61af66fc99e Initial load duke parents: diff changeset	1623 InitializeNode* init = alloc->initialization();
a61af66fc99e Initial load duke parents: diff changeset	1624 // Store the klass & mark bits
a61af66fc99e Initial load duke parents: diff changeset	1625 Node* mark_node = NULL;
a61af66fc99e Initial load duke parents: diff changeset	1626 // For now only enable fast locking for non-array types
a61af66fc99e Initial load duke parents: diff changeset	1627 if (UseBiasedLocking && (length == NULL)) {
4762 069ab3f976d3 7118863: Move sizeof(klassOopDesc) into the Klass::_offset_in_bytes() functions stefank parents: 4115 diff changeset	1628 mark_node = make_load(control, rawmem, klass_node, in_bytes(Klass::prototype_header_offset()), TypeRawPtr::BOTTOM, T_ADDRESS);
0 a61af66fc99e Initial load duke parents: diff changeset	1629 } else {
a61af66fc99e Initial load duke parents: diff changeset	1630 mark_node = makecon(TypeRawPtr::make((address)markOopDesc::prototype()));
a61af66fc99e Initial load duke parents: diff changeset	1631 }
a61af66fc99e Initial load duke parents: diff changeset	1632 rawmem = make_store(control, rawmem, object, oopDesc::mark_offset_in_bytes(), mark_node, T_ADDRESS);
113 ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 73 diff changeset	1633
6725 da91efe96a93 6964458: Reimplement class meta-data storage to use native memory coleenp parents: 6197 diff changeset	1634 rawmem = make_store(control, rawmem, object, oopDesc::klass_offset_in_bytes(), klass_node, T_METADATA);
0 a61af66fc99e Initial load duke parents: diff changeset	1635 int header_size = alloc->minimum_header_size(); // conservatively small
a61af66fc99e Initial load duke parents: diff changeset	1636
a61af66fc99e Initial load duke parents: diff changeset	1637 // Array length
a61af66fc99e Initial load duke parents: diff changeset	1638 if (length != NULL) { // Arrays need length field
a61af66fc99e Initial load duke parents: diff changeset	1639 rawmem = make_store(control, rawmem, object, arrayOopDesc::length_offset_in_bytes(), length, T_INT);
a61af66fc99e Initial load duke parents: diff changeset	1640 // conservatively small header size:
113 ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 73 diff changeset	1641 header_size = arrayOopDesc::base_offset_in_bytes(T_BYTE);
0 a61af66fc99e Initial load duke parents: diff changeset	1642 ciKlass* k = _igvn.type(klass_node)->is_klassptr()->klass();
a61af66fc99e Initial load duke parents: diff changeset	1643 if (k->is_array_klass()) // we know the exact header size in most cases:
a61af66fc99e Initial load duke parents: diff changeset	1644 header_size = Klass::layout_helper_header_size(k->layout_helper());
a61af66fc99e Initial load duke parents: diff changeset	1645 }
a61af66fc99e Initial load duke parents: diff changeset	1646
a61af66fc99e Initial load duke parents: diff changeset	1647 // Clear the object body, if necessary.
a61af66fc99e Initial load duke parents: diff changeset	1648 if (init == NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1649 // The init has somehow disappeared; be cautious and clear everything.
a61af66fc99e Initial load duke parents: diff changeset	1650 //
a61af66fc99e Initial load duke parents: diff changeset	1651 // This can happen if a node is allocated but an uncommon trap occurs
a61af66fc99e Initial load duke parents: diff changeset	1652 // immediately. In this case, the Initialize gets associated with the
a61af66fc99e Initial load duke parents: diff changeset	1653 // trap, and may be placed in a different (outer) loop, if the Allocate
a61af66fc99e Initial load duke parents: diff changeset	1654 // is in a loop. If (this is rare) the inner loop gets unrolled, then
a61af66fc99e Initial load duke parents: diff changeset	1655 // there can be two Allocates to one Initialize. The answer in all these
a61af66fc99e Initial load duke parents: diff changeset	1656 // edge cases is safety first. It is always safe to clear immediately
a61af66fc99e Initial load duke parents: diff changeset	1657 // within an Allocate, and then (maybe or maybe not) clear some more later.
a61af66fc99e Initial load duke parents: diff changeset	1658 if (!ZeroTLAB)
a61af66fc99e Initial load duke parents: diff changeset	1659 rawmem = ClearArrayNode::clear_memory(control, rawmem, object,
a61af66fc99e Initial load duke parents: diff changeset	1660 header_size, size_in_bytes,
a61af66fc99e Initial load duke parents: diff changeset	1661 &_igvn);
a61af66fc99e Initial load duke parents: diff changeset	1662 } else {
a61af66fc99e Initial load duke parents: diff changeset	1663 if (!init->is_complete()) {
a61af66fc99e Initial load duke parents: diff changeset	1664 // Try to win by zeroing only what the init does not store.
a61af66fc99e Initial load duke parents: diff changeset	1665 // We can also try to do some peephole optimizations,
a61af66fc99e Initial load duke parents: diff changeset	1666 // such as combining some adjacent subword stores.
a61af66fc99e Initial load duke parents: diff changeset	1667 rawmem = init->complete_stores(control, rawmem, object,
a61af66fc99e Initial load duke parents: diff changeset	1668 header_size, size_in_bytes, &_igvn);
a61af66fc99e Initial load duke parents: diff changeset	1669 }
a61af66fc99e Initial load duke parents: diff changeset	1670 // We have no more use for this link, since the AllocateNode goes away:
a61af66fc99e Initial load duke parents: diff changeset	1671 init->set_req(InitializeNode::RawAddress, top());
a61af66fc99e Initial load duke parents: diff changeset	1672 // (If we keep the link, it just confuses the register allocator,
a61af66fc99e Initial load duke parents: diff changeset	1673 // who thinks he sees a real use of the address by the membar.)
a61af66fc99e Initial load duke parents: diff changeset	1674 }
a61af66fc99e Initial load duke parents: diff changeset	1675
a61af66fc99e Initial load duke parents: diff changeset	1676 return rawmem;
a61af66fc99e Initial load duke parents: diff changeset	1677 }
a61af66fc99e Initial load duke parents: diff changeset	1678
a61af66fc99e Initial load duke parents: diff changeset	1679 // Generate prefetch instructions for next allocations.
a61af66fc99e Initial load duke parents: diff changeset	1680 Node* PhaseMacroExpand::prefetch_allocation(Node* i_o, Node*& needgc_false,
a61af66fc99e Initial load duke parents: diff changeset	1681 Node*& contended_phi_rawmem,
a61af66fc99e Initial load duke parents: diff changeset	1682 Node* old_eden_top, Node* new_eden_top,
a61af66fc99e Initial load duke parents: diff changeset	1683 Node* length) {
1367 9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1684 enum { fall_in_path = 1, pf_path = 2 };
0 a61af66fc99e Initial load duke parents: diff changeset	1685 if( UseTLAB && AllocatePrefetchStyle == 2 ) {
a61af66fc99e Initial load duke parents: diff changeset	1686 // Generate prefetch allocation with watermark check.
a61af66fc99e Initial load duke parents: diff changeset	1687 // As an allocation hits the watermark, we will prefetch starting
a61af66fc99e Initial load duke parents: diff changeset	1688 // at a "distance" away from watermark.
a61af66fc99e Initial load duke parents: diff changeset	1689
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1690 Node *pf_region = new (C) RegionNode(3);
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1691 Node *pf_phi_rawmem = new (C) PhiNode( pf_region, Type::MEMORY,
0 a61af66fc99e Initial load duke parents: diff changeset	1692 TypeRawPtr::BOTTOM );
a61af66fc99e Initial load duke parents: diff changeset	1693 // I/O is used for Prefetch
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1694 Node *pf_phi_abio = new (C) PhiNode( pf_region, Type::ABIO );
0 a61af66fc99e Initial load duke parents: diff changeset	1695
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1696 Node *thread = new (C) ThreadLocalNode();
0 a61af66fc99e Initial load duke parents: diff changeset	1697 transform_later(thread);
a61af66fc99e Initial load duke parents: diff changeset	1698
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1699 Node eden_pf_adr = new (C) AddPNode( top()/not oop*/, thread,
0 a61af66fc99e Initial load duke parents: diff changeset	1700 _igvn.MakeConX(in_bytes(JavaThread::tlab_pf_top_offset())) );
a61af66fc99e Initial load duke parents: diff changeset	1701 transform_later(eden_pf_adr);
a61af66fc99e Initial load duke parents: diff changeset	1702
14429 2113136690bc 8024921: PPC64 (part 113): Extend Load and Store nodes to know about memory ordering goetz parents: 12226 diff changeset	1703 Node *old_pf_wm = new (C) LoadPNode(needgc_false,
0 a61af66fc99e Initial load duke parents: diff changeset	1704 contended_phi_rawmem, eden_pf_adr,
14429 2113136690bc 8024921: PPC64 (part 113): Extend Load and Store nodes to know about memory ordering goetz parents: 12226 diff changeset	1705 TypeRawPtr::BOTTOM, TypeRawPtr::BOTTOM,
2113136690bc 8024921: PPC64 (part 113): Extend Load and Store nodes to know about memory ordering goetz parents: 12226 diff changeset	1706 MemNode::unordered);
0 a61af66fc99e Initial load duke parents: diff changeset	1707 transform_later(old_pf_wm);
a61af66fc99e Initial load duke parents: diff changeset	1708
a61af66fc99e Initial load duke parents: diff changeset	1709 // check against new_eden_top
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1710 Node *need_pf_cmp = new (C) CmpPNode( new_eden_top, old_pf_wm );
0 a61af66fc99e Initial load duke parents: diff changeset	1711 transform_later(need_pf_cmp);
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1712 Node *need_pf_bol = new (C) BoolNode( need_pf_cmp, BoolTest::ge );
0 a61af66fc99e Initial load duke parents: diff changeset	1713 transform_later(need_pf_bol);
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1714 IfNode *need_pf_iff = new (C) IfNode( needgc_false, need_pf_bol,
0 a61af66fc99e Initial load duke parents: diff changeset	1715 PROB_UNLIKELY_MAG(4), COUNT_UNKNOWN );
a61af66fc99e Initial load duke parents: diff changeset	1716 transform_later(need_pf_iff);
a61af66fc99e Initial load duke parents: diff changeset	1717
a61af66fc99e Initial load duke parents: diff changeset	1718 // true node, add prefetchdistance
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1719 Node *need_pf_true = new (C) IfTrueNode( need_pf_iff );
0 a61af66fc99e Initial load duke parents: diff changeset	1720 transform_later(need_pf_true);
a61af66fc99e Initial load duke parents: diff changeset	1721
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1722 Node *need_pf_false = new (C) IfFalseNode( need_pf_iff );
0 a61af66fc99e Initial load duke parents: diff changeset	1723 transform_later(need_pf_false);
a61af66fc99e Initial load duke parents: diff changeset	1724
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1725 Node *new_pf_wmt = new (C) AddPNode( top(), old_pf_wm,
0 a61af66fc99e Initial load duke parents: diff changeset	1726 _igvn.MakeConX(AllocatePrefetchDistance) );
a61af66fc99e Initial load duke parents: diff changeset	1727 transform_later(new_pf_wmt );
a61af66fc99e Initial load duke parents: diff changeset	1728 new_pf_wmt->set_req(0, need_pf_true);
a61af66fc99e Initial load duke parents: diff changeset	1729
14429 2113136690bc 8024921: PPC64 (part 113): Extend Load and Store nodes to know about memory ordering goetz parents: 12226 diff changeset	1730 Node *store_new_wmt = new (C) StorePNode(need_pf_true,
0 a61af66fc99e Initial load duke parents: diff changeset	1731 contended_phi_rawmem, eden_pf_adr,
14429 2113136690bc 8024921: PPC64 (part 113): Extend Load and Store nodes to know about memory ordering goetz parents: 12226 diff changeset	1732 TypeRawPtr::BOTTOM, new_pf_wmt,
2113136690bc 8024921: PPC64 (part 113): Extend Load and Store nodes to know about memory ordering goetz parents: 12226 diff changeset	1733 MemNode::unordered);
0 a61af66fc99e Initial load duke parents: diff changeset	1734 transform_later(store_new_wmt);
a61af66fc99e Initial load duke parents: diff changeset	1735
a61af66fc99e Initial load duke parents: diff changeset	1736 // adding prefetches
a61af66fc99e Initial load duke parents: diff changeset	1737 pf_phi_abio->init_req( fall_in_path, i_o );
a61af66fc99e Initial load duke parents: diff changeset	1738
a61af66fc99e Initial load duke parents: diff changeset	1739 Node *prefetch_adr;
a61af66fc99e Initial load duke parents: diff changeset	1740 Node *prefetch;
a61af66fc99e Initial load duke parents: diff changeset	1741 uint lines = AllocatePrefetchDistance / AllocatePrefetchStepSize;
a61af66fc99e Initial load duke parents: diff changeset	1742 uint step_size = AllocatePrefetchStepSize;
a61af66fc99e Initial load duke parents: diff changeset	1743 uint distance = 0;
a61af66fc99e Initial load duke parents: diff changeset	1744
a61af66fc99e Initial load duke parents: diff changeset	1745 for ( uint i = 0; i < lines; i++ ) {
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1746 prefetch_adr = new (C) AddPNode( old_pf_wm, new_pf_wmt,
0 a61af66fc99e Initial load duke parents: diff changeset	1747 _igvn.MakeConX(distance) );
a61af66fc99e Initial load duke parents: diff changeset	1748 transform_later(prefetch_adr);
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1749 prefetch = new (C) PrefetchAllocationNode( i_o, prefetch_adr );
0 a61af66fc99e Initial load duke parents: diff changeset	1750 transform_later(prefetch);
a61af66fc99e Initial load duke parents: diff changeset	1751 distance += step_size;
a61af66fc99e Initial load duke parents: diff changeset	1752 i_o = prefetch;
a61af66fc99e Initial load duke parents: diff changeset	1753 }
a61af66fc99e Initial load duke parents: diff changeset	1754 pf_phi_abio->set_req( pf_path, i_o );
a61af66fc99e Initial load duke parents: diff changeset	1755
a61af66fc99e Initial load duke parents: diff changeset	1756 pf_region->init_req( fall_in_path, need_pf_false );
a61af66fc99e Initial load duke parents: diff changeset	1757 pf_region->init_req( pf_path, need_pf_true );
a61af66fc99e Initial load duke parents: diff changeset	1758
a61af66fc99e Initial load duke parents: diff changeset	1759 pf_phi_rawmem->init_req( fall_in_path, contended_phi_rawmem );
a61af66fc99e Initial load duke parents: diff changeset	1760 pf_phi_rawmem->init_req( pf_path, store_new_wmt );
a61af66fc99e Initial load duke parents: diff changeset	1761
a61af66fc99e Initial load duke parents: diff changeset	1762 transform_later(pf_region);
a61af66fc99e Initial load duke parents: diff changeset	1763 transform_later(pf_phi_rawmem);
a61af66fc99e Initial load duke parents: diff changeset	1764 transform_later(pf_phi_abio);
a61af66fc99e Initial load duke parents: diff changeset	1765
a61af66fc99e Initial load duke parents: diff changeset	1766 needgc_false = pf_region;
a61af66fc99e Initial load duke parents: diff changeset	1767 contended_phi_rawmem = pf_phi_rawmem;
a61af66fc99e Initial load duke parents: diff changeset	1768 i_o = pf_phi_abio;
1367 9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1769 } else if( UseTLAB && AllocatePrefetchStyle == 3 ) {
3854 1af104d6cf99 7079329: Adjust allocation prefetching for T4 kvn parents: 3849 diff changeset	1770 // Insert a prefetch for each allocation.
1af104d6cf99 7079329: Adjust allocation prefetching for T4 kvn parents: 3849 diff changeset	1771 // This code is used for Sparc with BIS.
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1772 Node *pf_region = new (C) RegionNode(3);
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1773 Node *pf_phi_rawmem = new (C) PhiNode( pf_region, Type::MEMORY,
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1774 TypeRawPtr::BOTTOM );
1367 9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1775
3854 1af104d6cf99 7079329: Adjust allocation prefetching for T4 kvn parents: 3849 diff changeset	1776 // Generate several prefetch instructions.
1af104d6cf99 7079329: Adjust allocation prefetching for T4 kvn parents: 3849 diff changeset	1777 uint lines = (length != NULL) ? AllocatePrefetchLines : AllocateInstancePrefetchLines;
1367 9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1778 uint step_size = AllocatePrefetchStepSize;
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1779 uint distance = AllocatePrefetchDistance;
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1780
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1781 // Next cache address.
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1782 Node *cache_adr = new (C) AddPNode(old_eden_top, old_eden_top,
1367 9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1783 _igvn.MakeConX(distance));
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1784 transform_later(cache_adr);
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1785 cache_adr = new (C) CastP2XNode(needgc_false, cache_adr);
1367 9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1786 transform_later(cache_adr);
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1787 Node* mask = _igvn.MakeConX(~(intptr_t)(step_size-1));
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1788 cache_adr = new (C) AndXNode(cache_adr, mask);
1367 9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1789 transform_later(cache_adr);
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1790 cache_adr = new (C) CastX2PNode(cache_adr);
1367 9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1791 transform_later(cache_adr);
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1792
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1793 // Prefetch
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1794 Node *prefetch = new (C) PrefetchAllocationNode( contended_phi_rawmem, cache_adr );
1367 9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1795 prefetch->set_req(0, needgc_false);
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1796 transform_later(prefetch);
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1797 contended_phi_rawmem = prefetch;
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1798 Node *prefetch_adr;
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1799 distance = step_size;
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1800 for ( uint i = 1; i < lines; i++ ) {
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1801 prefetch_adr = new (C) AddPNode( cache_adr, cache_adr,
1367 9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1802 _igvn.MakeConX(distance) );
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1803 transform_later(prefetch_adr);
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1804 prefetch = new (C) PrefetchAllocationNode( contended_phi_rawmem, prefetch_adr );
1367 9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1805 transform_later(prefetch);
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1806 distance += step_size;
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1807 contended_phi_rawmem = prefetch;
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1808 }
0 a61af66fc99e Initial load duke parents: diff changeset	1809 } else if( AllocatePrefetchStyle > 0 ) {
a61af66fc99e Initial load duke parents: diff changeset	1810 // Insert a prefetch for each allocation only on the fast-path
a61af66fc99e Initial load duke parents: diff changeset	1811 Node *prefetch_adr;
a61af66fc99e Initial load duke parents: diff changeset	1812 Node *prefetch;
3854 1af104d6cf99 7079329: Adjust allocation prefetching for T4 kvn parents: 3849 diff changeset	1813 // Generate several prefetch instructions.
1af104d6cf99 7079329: Adjust allocation prefetching for T4 kvn parents: 3849 diff changeset	1814 uint lines = (length != NULL) ? AllocatePrefetchLines : AllocateInstancePrefetchLines;
0 a61af66fc99e Initial load duke parents: diff changeset	1815 uint step_size = AllocatePrefetchStepSize;
a61af66fc99e Initial load duke parents: diff changeset	1816 uint distance = AllocatePrefetchDistance;
a61af66fc99e Initial load duke parents: diff changeset	1817 for ( uint i = 0; i < lines; i++ ) {
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1818 prefetch_adr = new (C) AddPNode( old_eden_top, new_eden_top,
0 a61af66fc99e Initial load duke parents: diff changeset	1819 _igvn.MakeConX(distance) );
a61af66fc99e Initial load duke parents: diff changeset	1820 transform_later(prefetch_adr);
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	1821 prefetch = new (C) PrefetchAllocationNode( i_o, prefetch_adr );
0 a61af66fc99e Initial load duke parents: diff changeset	1822 // Do not let it float too high, since if eden_top == eden_end,
a61af66fc99e Initial load duke parents: diff changeset	1823 // both might be null.
a61af66fc99e Initial load duke parents: diff changeset	1824 if( i == 0 ) { // Set control for first prefetch, next follows it
a61af66fc99e Initial load duke parents: diff changeset	1825 prefetch->init_req(0, needgc_false);
a61af66fc99e Initial load duke parents: diff changeset	1826 }
a61af66fc99e Initial load duke parents: diff changeset	1827 transform_later(prefetch);
a61af66fc99e Initial load duke parents: diff changeset	1828 distance += step_size;
a61af66fc99e Initial load duke parents: diff changeset	1829 i_o = prefetch;
a61af66fc99e Initial load duke parents: diff changeset	1830 }
a61af66fc99e Initial load duke parents: diff changeset	1831 }
a61af66fc99e Initial load duke parents: diff changeset	1832 return i_o;
a61af66fc99e Initial load duke parents: diff changeset	1833 }
a61af66fc99e Initial load duke parents: diff changeset	1834
a61af66fc99e Initial load duke parents: diff changeset	1835
a61af66fc99e Initial load duke parents: diff changeset	1836 void PhaseMacroExpand::expand_allocate(AllocateNode *alloc) {
a61af66fc99e Initial load duke parents: diff changeset	1837 expand_allocate_common(alloc, NULL,
a61af66fc99e Initial load duke parents: diff changeset	1838 OptoRuntime::new_instance_Type(),
a61af66fc99e Initial load duke parents: diff changeset	1839 OptoRuntime::new_instance_Java());
a61af66fc99e Initial load duke parents: diff changeset	1840 }
a61af66fc99e Initial load duke parents: diff changeset	1841
a61af66fc99e Initial load duke parents: diff changeset	1842 void PhaseMacroExpand::expand_allocate_array(AllocateArrayNode *alloc) {
a61af66fc99e Initial load duke parents: diff changeset	1843 Node* length = alloc->in(AllocateNode::ALength);
3961 a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1844 InitializeNode* init = alloc->initialization();
a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1845 Node* klass_node = alloc->in(AllocateNode::KlassNode);
a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1846 ciKlass* k = _igvn.type(klass_node)->is_klassptr()->klass();
a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1847 address slow_call_address; // Address of slow call
a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1848 if (init != NULL && init->is_complete_with_arraycopy() &&
a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1849 k->is_type_array_klass()) {
a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1850 // Don't zero type array during slow allocation in VM since
a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1851 // it will be initialized later by arraycopy in compiled code.
a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1852 slow_call_address = OptoRuntime::new_array_nozero_Java();
a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1853 } else {
a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1854 slow_call_address = OptoRuntime::new_array_Java();
a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1855 }
0 a61af66fc99e Initial load duke parents: diff changeset	1856 expand_allocate_common(alloc, length,
a61af66fc99e Initial load duke parents: diff changeset	1857 OptoRuntime::new_array_Type(),
3961 a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1858 slow_call_address);
0 a61af66fc99e Initial load duke parents: diff changeset	1859 }
a61af66fc99e Initial load duke parents: diff changeset	1860
4777 e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1861 //-------------------mark_eliminated_box----------------------------------
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1862 //
3754 642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1863 // During EA obj may point to several objects but after few ideal graph
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1864 // transformations (CCP) it may point to only one non escaping object
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1865 // (but still using phi), corresponding locks and unlocks will be marked
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1866 // for elimination. Later obj could be replaced with a new node (new phi)
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1867 // and which does not have escape information. And later after some graph
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1868 // reshape other locks and unlocks (which were not marked for elimination
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1869 // before) are connected to this new obj (phi) but they still will not be
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1870 // marked for elimination since new obj has no escape information.
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1871 // Mark all associated (same box and obj) lock and unlock nodes for
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1872 // elimination if some of them marked already.
4777 e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1873 void PhaseMacroExpand::mark_eliminated_box(Node* oldbox, Node* obj) {
4792 89d0a5d40008 7129618: assert(obj_node->eqv_uncast(obj),""); kvn parents: 4790 diff changeset	1874 if (oldbox->as_BoxLock()->is_eliminated())
89d0a5d40008 7129618: assert(obj_node->eqv_uncast(obj),""); kvn parents: 4790 diff changeset	1875 return; // This BoxLock node was processed already.
4777 e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1876
4792 89d0a5d40008 7129618: assert(obj_node->eqv_uncast(obj),""); kvn parents: 4790 diff changeset	1877 // New implementation (EliminateNestedLocks) has separate BoxLock
89d0a5d40008 7129618: assert(obj_node->eqv_uncast(obj),""); kvn parents: 4790 diff changeset	1878 // node for each locked region so mark all associated locks/unlocks as
89d0a5d40008 7129618: assert(obj_node->eqv_uncast(obj),""); kvn parents: 4790 diff changeset	1879 // eliminated even if different objects are referenced in one locked region
89d0a5d40008 7129618: assert(obj_node->eqv_uncast(obj),""); kvn parents: 4790 diff changeset	1880 // (for example, OSR compilation of nested loop inside locked scope).
89d0a5d40008 7129618: assert(obj_node->eqv_uncast(obj),""); kvn parents: 4790 diff changeset	1881 if (EliminateNestedLocks \|\|
4777 e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1882 oldbox->as_BoxLock()->is_simple_lock_region(NULL, obj)) {
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1883 // Box is used only in one lock region. Mark this box as eliminated.
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1884 _igvn.hash_delete(oldbox);
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1885 oldbox->as_BoxLock()->set_eliminated(); // This changes box's hash value
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1886 _igvn.hash_insert(oldbox);
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1887
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1888 for (uint i = 0; i < oldbox->outcnt(); i++) {
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1889 Node* u = oldbox->raw_out(i);
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1890 if (u->is_AbstractLock() && !u->as_AbstractLock()->is_non_esc_obj()) {
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1891 AbstractLockNode* alock = u->as_AbstractLock();
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1892 // Check lock's box since box could be referenced by Lock's debug info.
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1893 if (alock->box_node() == oldbox) {
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1894 // Mark eliminated all related locks and unlocks.
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1895 alock->set_non_esc_obj();
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1896 }
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1897 }
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1898 }
3754 642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1899 return;
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1900 }
4777 e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1901
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1902 // Create new "eliminated" BoxLock node and use it in monitor debug info
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1903 // instead of oldbox for the same object.
4790 b0ff910edfc9 7128355: assert(!nocreate) failed: Cannot build a phi for a block already parsed kvn parents: 4778 diff changeset	1904 BoxLockNode* newbox = oldbox->clone()->as_BoxLock();
4777 e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1905
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1906 // Note: BoxLock node is marked eliminated only here and it is used
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1907 // to indicate that all associated lock and unlock nodes are marked
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1908 // for elimination.
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1909 newbox->set_eliminated();
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1910 transform_later(newbox);
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1911
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1912 // Replace old box node with new box for all users of the same object.
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1913 for (uint i = 0; i < oldbox->outcnt();) {
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1914 bool next_edge = true;
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1915
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1916 Node* u = oldbox->raw_out(i);
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1917 if (u->is_AbstractLock()) {
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1918 AbstractLockNode* alock = u->as_AbstractLock();
4778 35acf8f0a2e4 7128352: assert(obj_node == obj) failed kvn parents: 4777 diff changeset	1919 if (alock->box_node() == oldbox && alock->obj_node()->eqv_uncast(obj)) {
4777 e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1920 // Replace Box and mark eliminated all related locks and unlocks.
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1921 alock->set_non_esc_obj();
6144 5e990493719e 7173340: C2: code cleanup: use PhaseIterGVN::replace_edge(Node, int, Node) where applicable kvn parents: 4894 diff changeset	1922 _igvn.rehash_node_delayed(alock);
4777 e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1923 alock->set_box_node(newbox);
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1924 next_edge = false;
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1925 }
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1926 }
4778 35acf8f0a2e4 7128352: assert(obj_node == obj) failed kvn parents: 4777 diff changeset	1927 if (u->is_FastLock() && u->as_FastLock()->obj_node()->eqv_uncast(obj)) {
4777 e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1928 FastLockNode* flock = u->as_FastLock();
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1929 assert(flock->box_node() == oldbox, "sanity");
6144 5e990493719e 7173340: C2: code cleanup: use PhaseIterGVN::replace_edge(Node, int, Node) where applicable kvn parents: 4894 diff changeset	1930 _igvn.rehash_node_delayed(flock);
4777 e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1931 flock->set_box_node(newbox);
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1932 next_edge = false;
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1933 }
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1934
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1935 // Replace old box in monitor debug info.
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1936 if (u->is_SafePoint() && u->as_SafePoint()->jvms()) {
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1937 SafePointNode* sfn = u->as_SafePoint();
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1938 JVMState* youngest_jvms = sfn->jvms();
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1939 int max_depth = youngest_jvms->depth();
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1940 for (int depth = 1; depth <= max_depth; depth++) {
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1941 JVMState* jvms = youngest_jvms->of_depth(depth);
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1942 int num_mon = jvms->nof_monitors();
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1943 // Loop over monitors
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1944 for (int idx = 0; idx < num_mon; idx++) {
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1945 Node* obj_node = sfn->monitor_obj(jvms, idx);
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1946 Node* box_node = sfn->monitor_box(jvms, idx);
4778 35acf8f0a2e4 7128352: assert(obj_node == obj) failed kvn parents: 4777 diff changeset	1947 if (box_node == oldbox && obj_node->eqv_uncast(obj)) {
4777 e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1948 int j = jvms->monitor_box_offset(idx);
6144 5e990493719e 7173340: C2: code cleanup: use PhaseIterGVN::replace_edge(Node, int, Node) where applicable kvn parents: 4894 diff changeset	1949 _igvn.replace_input_of(u, j, newbox);
4777 e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1950 next_edge = false;
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1951 }
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1952 }
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1953 }
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1954 }
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1955 if (next_edge) i++;
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1956 }
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1957 }
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1958
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1959 //-----------------------mark_eliminated_locking_nodes-----------------------
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1960 void PhaseMacroExpand::mark_eliminated_locking_nodes(AbstractLockNode *alock) {
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1961 if (EliminateNestedLocks) {
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1962 if (alock->is_nested()) {
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1963 assert(alock->box_node()->as_BoxLock()->is_eliminated(), "sanity");
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1964 return;
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1965 } else if (!alock->is_non_esc_obj()) { // Not eliminated or coarsened
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1966 // Only Lock node has JVMState needed here.
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1967 if (alock->jvms() != NULL && alock->as_Lock()->is_nested_lock_region()) {
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1968 // Mark eliminated related nested locks and unlocks.
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1969 Node* obj = alock->obj_node();
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1970 BoxLockNode* box_node = alock->box_node()->as_BoxLock();
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1971 assert(!box_node->is_eliminated(), "should not be marked yet");
3754 642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1972 // Note: BoxLock node is marked eliminated only here
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1973 // and it is used to indicate that all associated lock
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1974 // and unlock nodes are marked for elimination.
4777 e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1975 box_node->set_eliminated(); // Box's hash is always NO_HASH here
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1976 for (uint i = 0; i < box_node->outcnt(); i++) {
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1977 Node* u = box_node->raw_out(i);
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1978 if (u->is_AbstractLock()) {
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1979 alock = u->as_AbstractLock();
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1980 if (alock->box_node() == box_node) {
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1981 // Verify that this Box is referenced only by related locks.
4778 35acf8f0a2e4 7128352: assert(obj_node == obj) failed kvn parents: 4777 diff changeset	1982 assert(alock->obj_node()->eqv_uncast(obj), "");
4777 e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1983 // Mark all related locks and unlocks.
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1984 alock->set_nested();
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1985 }
460 424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1986 }
4777 e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1987 }
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1988 }
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1989 return;
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1990 }
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1991 // Process locks for non escaping object
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1992 assert(alock->is_non_esc_obj(), "");
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1993 } // EliminateNestedLocks
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1994
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1995 if (alock->is_non_esc_obj()) { // Lock is used for non escaping object
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1996 // Look for all locks of this object and mark them and
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1997 // corresponding BoxLock nodes as eliminated.
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1998 Node* obj = alock->obj_node();
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	1999 for (uint j = 0; j < obj->outcnt(); j++) {
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	2000 Node* o = obj->raw_out(j);
4778 35acf8f0a2e4 7128352: assert(obj_node == obj) failed kvn parents: 4777 diff changeset	2001 if (o->is_AbstractLock() &&
35acf8f0a2e4 7128352: assert(obj_node == obj) failed kvn parents: 4777 diff changeset	2002 o->as_AbstractLock()->obj_node()->eqv_uncast(obj)) {
4777 e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	2003 alock = o->as_AbstractLock();
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	2004 Node* box = alock->box_node();
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	2005 // Replace old box node with new eliminated box for all users
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	2006 // of the same object and mark related locks as eliminated.
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	2007 mark_eliminated_box(box, obj);
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	2008 }
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	2009 }
e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	2010 }
3754 642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2011 }
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2012
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2013 // we have determined that this lock/unlock can be eliminated, we simply
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2014 // eliminate the node without expanding it.
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2015 //
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2016 // Note: The membar's associated with the lock/unlock are currently not
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2017 // eliminated. This should be investigated as a future enhancement.
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2018 //
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2019 bool PhaseMacroExpand::eliminate_locking_node(AbstractLockNode *alock) {
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2020
3754 642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2021 if (!alock->is_eliminated()) {
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2022 return false;
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2023 }
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2024 #ifdef ASSERT
4777 e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	2025 if (!alock->is_coarsened()) {
3754 642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2026 // Check that new "eliminated" BoxLock node is created.
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2027 BoxLockNode* oldbox = alock->box_node()->as_BoxLock();
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2028 assert(oldbox->is_eliminated(), "should be done already");
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2029 }
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2030 #endif
1080 7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	2031 CompileLog* log = C->log();
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	2032 if (log != NULL) {
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	2033 log->head("eliminate_lock lock='%d'",
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	2034 alock->is_Lock());
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	2035 JVMState* p = alock->jvms();
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	2036 while (p != NULL) {
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	2037 log->elem("jvms bci='%d' method='%d'", p->bci(), log->identify(p->method()));
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	2038 p = p->caller();
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	2039 }
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	2040 log->tail("eliminate_lock");
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	2041 }
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	2042
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2043 #ifndef PRODUCT
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2044 if (PrintEliminateLocks) {
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2045 if (alock->is_Lock()) {
4115 1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2046 tty->print_cr("++++ Eliminated: %d Lock", alock->_idx);
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2047 } else {
4115 1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2048 tty->print_cr("++++ Eliminated: %d Unlock", alock->_idx);
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2049 }
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2050 }
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2051 #endif
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2052
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2053 Node* mem = alock->in(TypeFunc::Memory);
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2054 Node* ctrl = alock->in(TypeFunc::Control);
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2055
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2056 extract_call_projections(alock);
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2057 // There are 2 projections from the lock. The lock node will
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2058 // be deleted when its last use is subsumed below.
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2059 assert(alock->outcnt() == 2 &&
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2060 _fallthroughproj != NULL &&
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2061 _memproj_fallthrough != NULL,
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2062 "Unexpected projections from Lock/Unlock");
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2063
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2064 Node* fallthroughproj = _fallthroughproj;
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2065 Node* memproj_fallthrough = _memproj_fallthrough;
0 a61af66fc99e Initial load duke parents: diff changeset	2066
a61af66fc99e Initial load duke parents: diff changeset	2067 // The memory projection from a lock/unlock is RawMem
a61af66fc99e Initial load duke parents: diff changeset	2068 // The input to a Lock is merged memory, so extract its RawMem input
a61af66fc99e Initial load duke parents: diff changeset	2069 // (unless the MergeMem has been optimized away.)
a61af66fc99e Initial load duke parents: diff changeset	2070 if (alock->is_Lock()) {
3849 f1c12354c3f7 7074017: Introduce MemBarAcquireLock/MemBarReleaseLock nodes for monitor enter/exit code paths roland parents: 3788 diff changeset	2071 // Seach for MemBarAcquireLock node and delete it also.
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2072 MemBarNode* membar = fallthroughproj->unique_ctrl_out()->as_MemBar();
3849 f1c12354c3f7 7074017: Introduce MemBarAcquireLock/MemBarReleaseLock nodes for monitor enter/exit code paths roland parents: 3788 diff changeset	2073 assert(membar != NULL && membar->Opcode() == Op_MemBarAcquireLock, "");
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2074 Node* ctrlproj = membar->proj_out(TypeFunc::Control);
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2075 Node* memproj = membar->proj_out(TypeFunc::Memory);
708 f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	2076 _igvn.replace_node(ctrlproj, fallthroughproj);
f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	2077 _igvn.replace_node(memproj, memproj_fallthrough);
460 424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2078
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2079 // Delete FastLock node also if this Lock node is unique user
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2080 // (a loop peeling may clone a Lock node).
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2081 Node* flock = alock->as_Lock()->fastlock_node();
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2082 if (flock->outcnt() == 1) {
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2083 assert(flock->unique_out() == alock, "sanity");
708 f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	2084 _igvn.replace_node(flock, top());
460 424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2085 }
0 a61af66fc99e Initial load duke parents: diff changeset	2086 }
a61af66fc99e Initial load duke parents: diff changeset	2087
3849 f1c12354c3f7 7074017: Introduce MemBarAcquireLock/MemBarReleaseLock nodes for monitor enter/exit code paths roland parents: 3788 diff changeset	2088 // Seach for MemBarReleaseLock node and delete it also.
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2089 if (alock->is_Unlock() && ctrl != NULL && ctrl->is_Proj() &&
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2090 ctrl->in(0)->is_MemBar()) {
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2091 MemBarNode* membar = ctrl->in(0)->as_MemBar();
3849 f1c12354c3f7 7074017: Introduce MemBarAcquireLock/MemBarReleaseLock nodes for monitor enter/exit code paths roland parents: 3788 diff changeset	2092 assert(membar->Opcode() == Op_MemBarReleaseLock &&
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2093 mem->is_Proj() && membar == mem->in(0), "");
708 f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	2094 _igvn.replace_node(fallthroughproj, ctrl);
f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	2095 _igvn.replace_node(memproj_fallthrough, mem);
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2096 fallthroughproj = ctrl;
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2097 memproj_fallthrough = mem;
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2098 ctrl = membar->in(TypeFunc::Control);
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2099 mem = membar->in(TypeFunc::Memory);
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2100 }
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2101
708 f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	2102 _igvn.replace_node(fallthroughproj, ctrl);
f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	2103 _igvn.replace_node(memproj_fallthrough, mem);
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2104 return true;
0 a61af66fc99e Initial load duke parents: diff changeset	2105 }
a61af66fc99e Initial load duke parents: diff changeset	2106
a61af66fc99e Initial load duke parents: diff changeset	2107
a61af66fc99e Initial load duke parents: diff changeset	2108 //------------------------------expand_lock_node----------------------
a61af66fc99e Initial load duke parents: diff changeset	2109 void PhaseMacroExpand::expand_lock_node(LockNode *lock) {
a61af66fc99e Initial load duke parents: diff changeset	2110
a61af66fc99e Initial load duke parents: diff changeset	2111 Node* ctrl = lock->in(TypeFunc::Control);
a61af66fc99e Initial load duke parents: diff changeset	2112 Node* mem = lock->in(TypeFunc::Memory);
a61af66fc99e Initial load duke parents: diff changeset	2113 Node* obj = lock->obj_node();
a61af66fc99e Initial load duke parents: diff changeset	2114 Node* box = lock->box_node();
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2115 Node* flock = lock->fastlock_node();
0 a61af66fc99e Initial load duke parents: diff changeset	2116
4790 b0ff910edfc9 7128355: assert(!nocreate) failed: Cannot build a phi for a block already parsed kvn parents: 4778 diff changeset	2117 assert(!box->as_BoxLock()->is_eliminated(), "sanity");
4777 e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	2118
0 a61af66fc99e Initial load duke parents: diff changeset	2119 // Make the merge point
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2120 Node *region;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2121 Node *mem_phi;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2122 Node *slow_path;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2123
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2124 if (UseOptoBiasInlining) {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2125 /*
605 98cb887364d3 6810672: Comment typos twisti parents: 601 diff changeset	2126 * See the full description in MacroAssembler::biased_locking_enter().
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2127 *
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2128 * if( (mark_word & biased_lock_mask) == biased_lock_pattern ) {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2129 * // The object is biased.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2130 * proto_node = klass->prototype_header;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2131 * o_node = thread \| proto_node;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2132 * x_node = o_node ^ mark_word;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2133 * if( (x_node & ~age_mask) == 0 ) { // Biased to the current thread ?
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2134 * // Done.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2135 * } else {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2136 * if( (x_node & biased_lock_mask) != 0 ) {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2137 * // The klass's prototype header is no longer biased.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2138 * cas(&mark_word, mark_word, proto_node)
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2139 * goto cas_lock;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2140 * } else {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2141 * // The klass's prototype header is still biased.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2142 * if( (x_node & epoch_mask) != 0 ) { // Expired epoch?
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2143 * old = mark_word;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2144 * new = o_node;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2145 * } else {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2146 * // Different thread or anonymous biased.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2147 * old = mark_word & (epoch_mask \| age_mask \| biased_lock_mask);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2148 * new = thread \| old;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2149 * }
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2150 * // Try to rebias.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2151 * if( cas(&mark_word, old, new) == 0 ) {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2152 * // Done.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2153 * } else {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2154 * goto slow_path; // Failed.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2155 * }
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2156 * }
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2157 * }
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2158 * } else {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2159 * // The object is not biased.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2160 * cas_lock:
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2161 * if( FastLock(obj) == 0 ) {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2162 * // Done.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2163 * } else {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2164 * slow_path:
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2165 * OptoRuntime::complete_monitor_locking_Java(obj);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2166 * }
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2167 * }
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2168 */
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2169
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2170 region = new (C) RegionNode(5);
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2171 // create a Phi for the memory state
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2172 mem_phi = new (C) PhiNode( region, Type::MEMORY, TypeRawPtr::BOTTOM);
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2173
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2174 Node* fast_lock_region = new (C) RegionNode(3);
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2175 Node* fast_lock_mem_phi = new (C) PhiNode( fast_lock_region, Type::MEMORY, TypeRawPtr::BOTTOM);
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2176
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2177 // First, check mark word for the biased lock pattern.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2178 Node* mark_node = make_load(ctrl, mem, obj, oopDesc::mark_offset_in_bytes(), TypeX_X, TypeX_X->basic_type());
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2179
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2180 // Get fast path - mark word has the biased lock pattern.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2181 ctrl = opt_bits_test(ctrl, fast_lock_region, 1, mark_node,
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2182 markOopDesc::biased_lock_mask_in_place,
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2183 markOopDesc::biased_lock_pattern, true);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2184 // fast_lock_region->in(1) is set to slow path.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2185 fast_lock_mem_phi->init_req(1, mem);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2186
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2187 // Now check that the lock is biased to the current thread and has
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2188 // the same epoch and bias as Klass::_prototype_header.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2189
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2190 // Special-case a fresh allocation to avoid building nodes:
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2191 Node* klass_node = AllocateNode::Ideal_klass(obj, &_igvn);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2192 if (klass_node == NULL) {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2193 Node* k_adr = basic_plus_adr(obj, oopDesc::klass_offset_in_bytes());
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2194 klass_node = transform_later( LoadKlassNode::make(_igvn, mem, k_adr, _igvn.type(k_adr)->is_ptr()) );
490 dd70dd4c91de 6782820: Server VM fails with "unhandled implicit exception in compiled code" kvn parents: 460 diff changeset	2195 #ifdef _LP64
12226 7944aba7ba41 8015107: NPG: Use consistent naming for metaspace concepts ehelin parents: 12158 diff changeset	2196 if (UseCompressedClassPointers && klass_node->is_DecodeNKlass()) {
490 dd70dd4c91de 6782820: Server VM fails with "unhandled implicit exception in compiled code" kvn parents: 460 diff changeset	2197 assert(klass_node->in(1)->Opcode() == Op_LoadNKlass, "sanity");
dd70dd4c91de 6782820: Server VM fails with "unhandled implicit exception in compiled code" kvn parents: 460 diff changeset	2198 klass_node->in(1)->init_req(0, ctrl);
dd70dd4c91de 6782820: Server VM fails with "unhandled implicit exception in compiled code" kvn parents: 460 diff changeset	2199 } else
dd70dd4c91de 6782820: Server VM fails with "unhandled implicit exception in compiled code" kvn parents: 460 diff changeset	2200 #endif
dd70dd4c91de 6782820: Server VM fails with "unhandled implicit exception in compiled code" kvn parents: 460 diff changeset	2201 klass_node->init_req(0, ctrl);
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2202 }
4762 069ab3f976d3 7118863: Move sizeof(klassOopDesc) into the Klass::_offset_in_bytes() functions stefank parents: 4115 diff changeset	2203 Node *proto_node = make_load(ctrl, mem, klass_node, in_bytes(Klass::prototype_header_offset()), TypeX_X, TypeX_X->basic_type());
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2204
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2205 Node* thread = transform_later(new (C) ThreadLocalNode());
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2206 Node* cast_thread = transform_later(new (C) CastP2XNode(ctrl, thread));
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2207 Node* o_node = transform_later(new (C) OrXNode(cast_thread, proto_node));
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2208 Node* x_node = transform_later(new (C) XorXNode(o_node, mark_node));
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2209
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2210 // Get slow path - mark word does NOT match the value.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2211 Node* not_biased_ctrl = opt_bits_test(ctrl, region, 3, x_node,
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2212 (~markOopDesc::age_mask_in_place), 0);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2213 // region->in(3) is set to fast path - the object is biased to the current thread.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2214 mem_phi->init_req(3, mem);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2215
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2216
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2217 // Mark word does NOT match the value (thread \| Klass::_prototype_header).
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2218
0 a61af66fc99e Initial load duke parents: diff changeset	2219
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2220 // First, check biased pattern.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2221 // Get fast path - _prototype_header has the same biased lock pattern.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2222 ctrl = opt_bits_test(not_biased_ctrl, fast_lock_region, 2, x_node,
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2223 markOopDesc::biased_lock_mask_in_place, 0, true);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2224
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2225 not_biased_ctrl = fast_lock_region->in(2); // Slow path
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2226 // fast_lock_region->in(2) - the prototype header is no longer biased
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2227 // and we have to revoke the bias on this object.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2228 // We are going to try to reset the mark of this object to the prototype
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2229 // value and fall through to the CAS-based locking scheme.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2230 Node* adr = basic_plus_adr(obj, oopDesc::mark_offset_in_bytes());
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2231 Node* cas = new (C) StoreXConditionalNode(not_biased_ctrl, mem, adr,
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2232 proto_node, mark_node);
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2233 transform_later(cas);
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2234 Node* proj = transform_later( new (C) SCMemProjNode(cas));
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2235 fast_lock_mem_phi->init_req(2, proj);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2236
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2237
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2238 // Second, check epoch bits.
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2239 Node* rebiased_region = new (C) RegionNode(3);
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2240 Node* old_phi = new (C) PhiNode( rebiased_region, TypeX_X);
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2241 Node* new_phi = new (C) PhiNode( rebiased_region, TypeX_X);
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2242
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2243 // Get slow path - mark word does NOT match epoch bits.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2244 Node* epoch_ctrl = opt_bits_test(ctrl, rebiased_region, 1, x_node,
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2245 markOopDesc::epoch_mask_in_place, 0);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2246 // The epoch of the current bias is not valid, attempt to rebias the object
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2247 // toward the current thread.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2248 rebiased_region->init_req(2, epoch_ctrl);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2249 old_phi->init_req(2, mark_node);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2250 new_phi->init_req(2, o_node);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2251
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2252 // rebiased_region->in(1) is set to fast path.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2253 // The epoch of the current bias is still valid but we know
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2254 // nothing about the owner; it might be set or it might be clear.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2255 Node* cmask = MakeConX(markOopDesc::biased_lock_mask_in_place \|
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2256 markOopDesc::age_mask_in_place \|
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2257 markOopDesc::epoch_mask_in_place);
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2258 Node* old = transform_later(new (C) AndXNode(mark_node, cmask));
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2259 cast_thread = transform_later(new (C) CastP2XNode(ctrl, thread));
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2260 Node* new_mark = transform_later(new (C) OrXNode(cast_thread, old));
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2261 old_phi->init_req(1, old);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2262 new_phi->init_req(1, new_mark);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2263
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2264 transform_later(rebiased_region);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2265 transform_later(old_phi);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2266 transform_later(new_phi);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2267
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2268 // Try to acquire the bias of the object using an atomic operation.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2269 // If this fails we will go in to the runtime to revoke the object's bias.
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2270 cas = new (C) StoreXConditionalNode(rebiased_region, mem, adr,
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2271 new_phi, old_phi);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2272 transform_later(cas);
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2273 proj = transform_later( new (C) SCMemProjNode(cas));
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2274
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2275 // Get slow path - Failed to CAS.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2276 not_biased_ctrl = opt_bits_test(rebiased_region, region, 4, cas, 0, 0);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2277 mem_phi->init_req(4, proj);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2278 // region->in(4) is set to fast path - the object is rebiased to the current thread.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2279
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2280 // Failed to CAS.
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2281 slow_path = new (C) RegionNode(3);
e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2282 Node *slow_mem = new (C) PhiNode( slow_path, Type::MEMORY, TypeRawPtr::BOTTOM);
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2283
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2284 slow_path->init_req(1, not_biased_ctrl); // Capture slow-control
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2285 slow_mem->init_req(1, proj);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2286
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2287 // Call CAS-based locking scheme (FastLock node).
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2288
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2289 transform_later(fast_lock_region);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2290 transform_later(fast_lock_mem_phi);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2291
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2292 // Get slow path - FastLock failed to lock the object.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2293 ctrl = opt_bits_test(fast_lock_region, region, 2, flock, 0, 0);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2294 mem_phi->init_req(2, fast_lock_mem_phi);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2295 // region->in(2) is set to fast path - the object is locked to the current thread.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2296
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2297 slow_path->init_req(2, ctrl); // Capture slow-control
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2298 slow_mem->init_req(2, fast_lock_mem_phi);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2299
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2300 transform_later(slow_path);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2301 transform_later(slow_mem);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2302 // Reset lock's memory edge.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2303 lock->set_req(TypeFunc::Memory, slow_mem);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2304
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2305 } else {
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2306 region = new (C) RegionNode(3);
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2307 // create a Phi for the memory state
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2308 mem_phi = new (C) PhiNode( region, Type::MEMORY, TypeRawPtr::BOTTOM);
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2309
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2310 // Optimize test; set region slot 2
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2311 slow_path = opt_bits_test(ctrl, region, 2, flock, 0, 0);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2312 mem_phi->init_req(2, mem);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2313 }
0 a61af66fc99e Initial load duke parents: diff changeset	2314
a61af66fc99e Initial load duke parents: diff changeset	2315 // Make slow path call
a61af66fc99e Initial load duke parents: diff changeset	2316 CallNode call = make_slow_call( (CallNode ) lock, OptoRuntime::complete_monitor_enter_Type(), OptoRuntime::complete_monitor_locking_Java(), NULL, slow_path, obj, box );
a61af66fc99e Initial load duke parents: diff changeset	2317
a61af66fc99e Initial load duke parents: diff changeset	2318 extract_call_projections(call);
a61af66fc99e Initial load duke parents: diff changeset	2319
a61af66fc99e Initial load duke parents: diff changeset	2320 // Slow path can only throw asynchronous exceptions, which are always
a61af66fc99e Initial load duke parents: diff changeset	2321 // de-opted. So the compiler thinks the slow-call can never throw an
a61af66fc99e Initial load duke parents: diff changeset	2322 // exception. If it DOES throw an exception we would need the debug
a61af66fc99e Initial load duke parents: diff changeset	2323 // info removed first (since if it throws there is no monitor).
a61af66fc99e Initial load duke parents: diff changeset	2324 assert ( _ioproj_fallthrough == NULL && _ioproj_catchall == NULL &&
a61af66fc99e Initial load duke parents: diff changeset	2325 _memproj_catchall == NULL && _catchallcatchproj == NULL, "Unexpected projection from Lock");
a61af66fc99e Initial load duke parents: diff changeset	2326
a61af66fc99e Initial load duke parents: diff changeset	2327 // Capture slow path
a61af66fc99e Initial load duke parents: diff changeset	2328 // disconnect fall-through projection from call and create a new one
a61af66fc99e Initial load duke parents: diff changeset	2329 // hook up users of fall-through projection to region
a61af66fc99e Initial load duke parents: diff changeset	2330 Node *slow_ctrl = _fallthroughproj->clone();
a61af66fc99e Initial load duke parents: diff changeset	2331 transform_later(slow_ctrl);
a61af66fc99e Initial load duke parents: diff changeset	2332 _igvn.hash_delete(_fallthroughproj);
7196 2aff40cb4703 7092905: C2: Keep track of the number of dead nodes bharadwaj parents: 6848 diff changeset	2333 _fallthroughproj->disconnect_inputs(NULL, C);
0 a61af66fc99e Initial load duke parents: diff changeset	2334 region->init_req(1, slow_ctrl);
a61af66fc99e Initial load duke parents: diff changeset	2335 // region inputs are now complete
a61af66fc99e Initial load duke parents: diff changeset	2336 transform_later(region);
708 f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	2337 _igvn.replace_node(_fallthroughproj, region);
0 a61af66fc99e Initial load duke parents: diff changeset	2338
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2339 Node *memproj = transform_later( new(C) ProjNode(call, TypeFunc::Memory) );
0 a61af66fc99e Initial load duke parents: diff changeset	2340 mem_phi->init_req(1, memproj );
a61af66fc99e Initial load duke parents: diff changeset	2341 transform_later(mem_phi);
708 f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	2342 _igvn.replace_node(_memproj_fallthrough, mem_phi);
0 a61af66fc99e Initial load duke parents: diff changeset	2343 }
a61af66fc99e Initial load duke parents: diff changeset	2344
a61af66fc99e Initial load duke parents: diff changeset	2345 //------------------------------expand_unlock_node----------------------
a61af66fc99e Initial load duke parents: diff changeset	2346 void PhaseMacroExpand::expand_unlock_node(UnlockNode *unlock) {
a61af66fc99e Initial load duke parents: diff changeset	2347
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2348 Node* ctrl = unlock->in(TypeFunc::Control);
0 a61af66fc99e Initial load duke parents: diff changeset	2349 Node* mem = unlock->in(TypeFunc::Memory);
a61af66fc99e Initial load duke parents: diff changeset	2350 Node* obj = unlock->obj_node();
a61af66fc99e Initial load duke parents: diff changeset	2351 Node* box = unlock->box_node();
a61af66fc99e Initial load duke parents: diff changeset	2352
4790 b0ff910edfc9 7128355: assert(!nocreate) failed: Cannot build a phi for a block already parsed kvn parents: 4778 diff changeset	2353 assert(!box->as_BoxLock()->is_eliminated(), "sanity");
4777 e9a5e0a812c8 7125896: Eliminate nested locks kvn parents: 4769 diff changeset	2354
0 a61af66fc99e Initial load duke parents: diff changeset	2355 // No need for a null check on unlock
a61af66fc99e Initial load duke parents: diff changeset	2356
a61af66fc99e Initial load duke parents: diff changeset	2357 // Make the merge point
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2358 Node *region;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2359 Node *mem_phi;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2360
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2361 if (UseOptoBiasInlining) {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2362 // Check for biased locking unlock case, which is a no-op.
605 98cb887364d3 6810672: Comment typos twisti parents: 601 diff changeset	2363 // See the full description in MacroAssembler::biased_locking_exit().
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2364 region = new (C) RegionNode(4);
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2365 // create a Phi for the memory state
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2366 mem_phi = new (C) PhiNode( region, Type::MEMORY, TypeRawPtr::BOTTOM);
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2367 mem_phi->init_req(3, mem);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2368
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2369 Node* mark_node = make_load(ctrl, mem, obj, oopDesc::mark_offset_in_bytes(), TypeX_X, TypeX_X->basic_type());
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2370 ctrl = opt_bits_test(ctrl, region, 3, mark_node,
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2371 markOopDesc::biased_lock_mask_in_place,
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2372 markOopDesc::biased_lock_pattern);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2373 } else {
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2374 region = new (C) RegionNode(3);
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2375 // create a Phi for the memory state
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2376 mem_phi = new (C) PhiNode( region, Type::MEMORY, TypeRawPtr::BOTTOM);
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2377 }
0 a61af66fc99e Initial load duke parents: diff changeset	2378
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2379 FastUnlockNode *funlock = new (C) FastUnlockNode( ctrl, obj, box );
0 a61af66fc99e Initial load duke parents: diff changeset	2380 funlock = transform_later( funlock )->as_FastUnlock();
a61af66fc99e Initial load duke parents: diff changeset	2381 // Optimize test; set region slot 2
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2382 Node *slow_path = opt_bits_test(ctrl, region, 2, funlock, 0, 0);
0 a61af66fc99e Initial load duke parents: diff changeset	2383
a61af66fc99e Initial load duke parents: diff changeset	2384 CallNode call = make_slow_call( (CallNode ) unlock, OptoRuntime::complete_monitor_exit_Type(), CAST_FROM_FN_PTR(address, SharedRuntime::complete_monitor_unlocking_C), "complete_monitor_unlocking_C", slow_path, obj, box );
a61af66fc99e Initial load duke parents: diff changeset	2385
a61af66fc99e Initial load duke parents: diff changeset	2386 extract_call_projections(call);
a61af66fc99e Initial load duke parents: diff changeset	2387
a61af66fc99e Initial load duke parents: diff changeset	2388 assert ( _ioproj_fallthrough == NULL && _ioproj_catchall == NULL &&
a61af66fc99e Initial load duke parents: diff changeset	2389 _memproj_catchall == NULL && _catchallcatchproj == NULL, "Unexpected projection from Lock");
a61af66fc99e Initial load duke parents: diff changeset	2390
a61af66fc99e Initial load duke parents: diff changeset	2391 // No exceptions for unlocking
a61af66fc99e Initial load duke parents: diff changeset	2392 // Capture slow path
a61af66fc99e Initial load duke parents: diff changeset	2393 // disconnect fall-through projection from call and create a new one
a61af66fc99e Initial load duke parents: diff changeset	2394 // hook up users of fall-through projection to region
a61af66fc99e Initial load duke parents: diff changeset	2395 Node *slow_ctrl = _fallthroughproj->clone();
a61af66fc99e Initial load duke parents: diff changeset	2396 transform_later(slow_ctrl);
a61af66fc99e Initial load duke parents: diff changeset	2397 _igvn.hash_delete(_fallthroughproj);
7196 2aff40cb4703 7092905: C2: Keep track of the number of dead nodes bharadwaj parents: 6848 diff changeset	2398 _fallthroughproj->disconnect_inputs(NULL, C);
0 a61af66fc99e Initial load duke parents: diff changeset	2399 region->init_req(1, slow_ctrl);
a61af66fc99e Initial load duke parents: diff changeset	2400 // region inputs are now complete
a61af66fc99e Initial load duke parents: diff changeset	2401 transform_later(region);
708 f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	2402 _igvn.replace_node(_fallthroughproj, region);
0 a61af66fc99e Initial load duke parents: diff changeset	2403
6804 e626685e9f6c 7193318: C2: remove number of inputs requirement from Node's new operator kvn parents: 6725 diff changeset	2404 Node *memproj = transform_later( new(C) ProjNode(call, TypeFunc::Memory) );
0 a61af66fc99e Initial load duke parents: diff changeset	2405 mem_phi->init_req(1, memproj );
a61af66fc99e Initial load duke parents: diff changeset	2406 mem_phi->init_req(2, mem);
a61af66fc99e Initial load duke parents: diff changeset	2407 transform_later(mem_phi);
708 f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	2408 _igvn.replace_node(_memproj_fallthrough, mem_phi);
0 a61af66fc99e Initial load duke parents: diff changeset	2409 }
a61af66fc99e Initial load duke parents: diff changeset	2410
4115 1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2411 //---------------------------eliminate_macro_nodes----------------------
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2412 // Eliminate scalar replaced allocations and associated locks.
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2413 void PhaseMacroExpand::eliminate_macro_nodes() {
0 a61af66fc99e Initial load duke parents: diff changeset	2414 if (C->macro_count() == 0)
4115 1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2415 return;
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2416
460 424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2417 // First, attempt to eliminate locks
3754 642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2418 int cnt = C->macro_count();
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2419 for (int i=0; i < cnt; i++) {
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2420 Node *n = C->macro_node(i);
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2421 if (n->is_AbstractLock()) { // Lock and Unlock nodes
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2422 // Before elimination mark all associated (same box and obj)
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2423 // lock and unlock nodes.
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2424 mark_eliminated_locking_nodes(n->as_AbstractLock());
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2425 }
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2426 }
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2427 bool progress = true;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2428 while (progress) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2429 progress = false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2430 for (int i = C->macro_count(); i > 0; i--) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2431 Node * n = C->macro_node(i-1);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2432 bool success = false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2433 debug_only(int old_macro_count = C->macro_count(););
460 424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2434 if (n->is_AbstractLock()) {
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2435 success = eliminate_locking_node(n->as_AbstractLock());
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2436 }
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2437 assert(success == (C->macro_count() < old_macro_count), "elimination reduces macro count");
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2438 progress = progress \|\| success;
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2439 }
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2440 }
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2441 // Next, attempt to eliminate allocations
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2442 progress = true;
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2443 while (progress) {
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2444 progress = false;
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2445 for (int i = C->macro_count(); i > 0; i--) {
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2446 Node * n = C->macro_node(i-1);
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2447 bool success = false;
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2448 debug_only(int old_macro_count = C->macro_count(););
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2449 switch (n->class_id()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2450 case Node::Class_Allocate:
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2451 case Node::Class_AllocateArray:
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2452 success = eliminate_allocate_node(n->as_Allocate());
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2453 break;
10278 6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	2454 case Node::Class_CallStaticJava:
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	2455 success = eliminate_boxing_node(n->as_CallStaticJava());
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	2456 break;
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2457 case Node::Class_Lock:
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2458 case Node::Class_Unlock:
460 424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2459 assert(!n->as_AbstractLock()->is_eliminated(), "sanity");
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2460 break;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2461 default:
4115 1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2462 assert(n->Opcode() == Op_LoopLimit \|\|
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2463 n->Opcode() == Op_Opaque1 \|\|
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2464 n->Opcode() == Op_Opaque2, "unknown node type in macro list");
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2465 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2466 assert(success == (C->macro_count() < old_macro_count), "elimination reduces macro count");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2467 progress = progress \|\| success;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2468 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2469 }
4115 1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2470 }
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2471
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2472 //------------------------------expand_macro_nodes----------------------
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2473 // Returns true if a failure occurred.
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2474 bool PhaseMacroExpand::expand_macro_nodes() {
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2475 // Last attempt to eliminate macro nodes.
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2476 eliminate_macro_nodes();
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2477
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2478 // Make sure expansion will not cause node limit to be exceeded.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2479 // Worst case is a macro node gets expanded into about 50 nodes.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2480 // Allow 50% more for optimization.
0 a61af66fc99e Initial load duke parents: diff changeset	2481 if (C->check_node_count(C->macro_count() * 75, "out of nodes before macro expansion" ) )
a61af66fc99e Initial load duke parents: diff changeset	2482 return true;
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2483
4115 1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2484 // Eliminate Opaque and LoopLimit nodes. Do it after all loop optimizations.
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2485 bool progress = true;
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2486 while (progress) {
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2487 progress = false;
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2488 for (int i = C->macro_count(); i > 0; i--) {
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2489 Node * n = C->macro_node(i-1);
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2490 bool success = false;
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2491 debug_only(int old_macro_count = C->macro_count(););
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2492 if (n->Opcode() == Op_LoopLimit) {
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2493 // Remove it from macro list and put on IGVN worklist to optimize.
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2494 C->remove_macro_node(n);
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2495 _igvn._worklist.push(n);
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2496 success = true;
10278 6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	2497 } else if (n->Opcode() == Op_CallStaticJava) {
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	2498 // Remove it from macro list and put on IGVN worklist to optimize.
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	2499 C->remove_macro_node(n);
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	2500 _igvn._worklist.push(n);
6f3fd5150b67 6934604: enable parts of EliminateAutoBox by default kvn parents: 8694 diff changeset	2501 success = true;
4115 1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2502 } else if (n->Opcode() == Op_Opaque1 \|\| n->Opcode() == Op_Opaque2) {
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2503 _igvn.replace_node(n, n->in(1));
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2504 success = true;
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2505 }
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2506 assert(success == (C->macro_count() < old_macro_count), "elimination reduces macro count");
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2507 progress = progress \|\| success;
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2508 }
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2509 }
1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2510
0 a61af66fc99e Initial load duke parents: diff changeset	2511 // expand "macro" nodes
a61af66fc99e Initial load duke parents: diff changeset	2512 // nodes are removed from the macro list as they are processed
a61af66fc99e Initial load duke parents: diff changeset	2513 while (C->macro_count() > 0) {
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2514 int macro_count = C->macro_count();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2515 Node * n = C->macro_node(macro_count-1);
0 a61af66fc99e Initial load duke parents: diff changeset	2516 assert(n->is_macro(), "only macro nodes expected here");
a61af66fc99e Initial load duke parents: diff changeset	2517 if (_igvn.type(n) == Type::TOP \|\| n->in(0)->is_top() ) {
a61af66fc99e Initial load duke parents: diff changeset	2518 // node is unreachable, so don't try to expand it
a61af66fc99e Initial load duke parents: diff changeset	2519 C->remove_macro_node(n);
a61af66fc99e Initial load duke parents: diff changeset	2520 continue;
a61af66fc99e Initial load duke parents: diff changeset	2521 }
a61af66fc99e Initial load duke parents: diff changeset	2522 switch (n->class_id()) {
a61af66fc99e Initial load duke parents: diff changeset	2523 case Node::Class_Allocate:
a61af66fc99e Initial load duke parents: diff changeset	2524 expand_allocate(n->as_Allocate());
a61af66fc99e Initial load duke parents: diff changeset	2525 break;
a61af66fc99e Initial load duke parents: diff changeset	2526 case Node::Class_AllocateArray:
a61af66fc99e Initial load duke parents: diff changeset	2527 expand_allocate_array(n->as_AllocateArray());
a61af66fc99e Initial load duke parents: diff changeset	2528 break;
a61af66fc99e Initial load duke parents: diff changeset	2529 case Node::Class_Lock:
a61af66fc99e Initial load duke parents: diff changeset	2530 expand_lock_node(n->as_Lock());
a61af66fc99e Initial load duke parents: diff changeset	2531 break;
a61af66fc99e Initial load duke parents: diff changeset	2532 case Node::Class_Unlock:
a61af66fc99e Initial load duke parents: diff changeset	2533 expand_unlock_node(n->as_Unlock());
a61af66fc99e Initial load duke parents: diff changeset	2534 break;
a61af66fc99e Initial load duke parents: diff changeset	2535 default:
a61af66fc99e Initial load duke parents: diff changeset	2536 assert(false, "unknown node type in macro list");
a61af66fc99e Initial load duke parents: diff changeset	2537 }
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2538 assert(C->macro_count() < macro_count, "must have deleted a node from macro list");
0 a61af66fc99e Initial load duke parents: diff changeset	2539 if (C->failing()) return true;
a61af66fc99e Initial load duke parents: diff changeset	2540 }
113 ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 73 diff changeset	2541
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 73 diff changeset	2542 _igvn.set_delay_transform(false);
0 a61af66fc99e Initial load duke parents: diff changeset	2543 _igvn.optimize();
4115 1bd45abaa507 6890673: Eliminate allocations immediately after EA kvn parents: 3961 diff changeset	2544 if (C->failing()) return true;
0 a61af66fc99e Initial load duke parents: diff changeset	2545 return false;
a61af66fc99e Initial load duke parents: diff changeset	2546 }

Mercurial > hg > truffle

annotate src/share/vm/opto/macro.cpp @ 14649:f6301b007a16