graal-compiler: src/share/vm/opto/macro.cpp annotate

annotate src/share/vm/opto/macro.cpp @ 3992:d1bdeef3e3e2

7098282: G1: assert(interval >= 0) failed: Sanity check, referencePolicy.cpp: 76 Summary: There is a race between one thread successfully forwarding and copying the klass mirror for the SoftReference class (including the static master clock) and another thread attempting to use the master clock while attempting to discover a soft reference object. Maintain a shadow copy of the soft reference master clock and use the shadow during reference discovery and reference processing. Reviewed-by: tonyp, brutisso, ysr

author	johnc
date	Wed, 12 Oct 2011 10:25:51 -0700
parents	a92cdbac8b9e
children	1bd45abaa507

rev	line source
0 a61af66fc99e Initial load duke parents: diff changeset	1 /*
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	2 * Copyright (c) 2005, 2011, Oracle and/or its affiliates. All rights reserved.
0 a61af66fc99e Initial load duke parents: diff changeset	3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
a61af66fc99e Initial load duke parents: diff changeset	4 *
a61af66fc99e Initial load duke parents: diff changeset	5 * This code is free software; you can redistribute it and/or modify it
a61af66fc99e Initial load duke parents: diff changeset	6 * under the terms of the GNU General Public License version 2 only, as
a61af66fc99e Initial load duke parents: diff changeset	7 * published by the Free Software Foundation.
a61af66fc99e Initial load duke parents: diff changeset	8 *
a61af66fc99e Initial load duke parents: diff changeset	9 * This code is distributed in the hope that it will be useful, but WITHOUT
a61af66fc99e Initial load duke parents: diff changeset	10 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
a61af66fc99e Initial load duke parents: diff changeset	11 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
a61af66fc99e Initial load duke parents: diff changeset	12 * version 2 for more details (a copy is included in the LICENSE file that
a61af66fc99e Initial load duke parents: diff changeset	13 * accompanied this code).
a61af66fc99e Initial load duke parents: diff changeset	14 *
a61af66fc99e Initial load duke parents: diff changeset	15 * You should have received a copy of the GNU General Public License version
a61af66fc99e Initial load duke parents: diff changeset	16 * 2 along with this work; if not, write to the Free Software Foundation,
a61af66fc99e Initial load duke parents: diff changeset	17 * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
a61af66fc99e Initial load duke parents: diff changeset	18 *
1552 c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1367 diff changeset	19 * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1367 diff changeset	20 * or visit www.oracle.com if you need additional information or have any
c18cbe5936b8 6941466: Oracle rebranding changes for Hotspot repositories trims parents: 1367 diff changeset	21 * questions.
0 a61af66fc99e Initial load duke parents: diff changeset	22 *
a61af66fc99e Initial load duke parents: diff changeset	23 */
a61af66fc99e Initial load duke parents: diff changeset	24
1972 f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	25 #include "precompiled.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	26 #include "compiler/compileLog.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	27 #include "libadt/vectset.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	28 #include "opto/addnode.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	29 #include "opto/callnode.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	30 #include "opto/cfgnode.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	31 #include "opto/compile.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	32 #include "opto/connode.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	33 #include "opto/locknode.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	34 #include "opto/loopnode.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	35 #include "opto/macro.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	36 #include "opto/memnode.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	37 #include "opto/node.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	38 #include "opto/phaseX.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	39 #include "opto/rootnode.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	40 #include "opto/runtime.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	41 #include "opto/subnode.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	42 #include "opto/type.hpp"
f95d63e2154a 6989984: Use standard include model for Hospot stefank parents: 1682 diff changeset	43 #include "runtime/sharedRuntime.hpp"
0 a61af66fc99e Initial load duke parents: diff changeset	44
a61af66fc99e Initial load duke parents: diff changeset	45
a61af66fc99e Initial load duke parents: diff changeset	46 //
a61af66fc99e Initial load duke parents: diff changeset	47 // Replace any references to "oldref" in inputs to "use" with "newref".
a61af66fc99e Initial load duke parents: diff changeset	48 // Returns the number of replacements made.
a61af66fc99e Initial load duke parents: diff changeset	49 //
a61af66fc99e Initial load duke parents: diff changeset	50 int PhaseMacroExpand::replace_input(Node use, Node oldref, Node *newref) {
a61af66fc99e Initial load duke parents: diff changeset	51 int nreplacements = 0;
a61af66fc99e Initial load duke parents: diff changeset	52 uint req = use->req();
a61af66fc99e Initial load duke parents: diff changeset	53 for (uint j = 0; j < use->len(); j++) {
a61af66fc99e Initial load duke parents: diff changeset	54 Node *uin = use->in(j);
a61af66fc99e Initial load duke parents: diff changeset	55 if (uin == oldref) {
a61af66fc99e Initial load duke parents: diff changeset	56 if (j < req)
a61af66fc99e Initial load duke parents: diff changeset	57 use->set_req(j, newref);
a61af66fc99e Initial load duke parents: diff changeset	58 else
a61af66fc99e Initial load duke parents: diff changeset	59 use->set_prec(j, newref);
a61af66fc99e Initial load duke parents: diff changeset	60 nreplacements++;
a61af66fc99e Initial load duke parents: diff changeset	61 } else if (j >= req && uin == NULL) {
a61af66fc99e Initial load duke parents: diff changeset	62 break;
a61af66fc99e Initial load duke parents: diff changeset	63 }
a61af66fc99e Initial load duke parents: diff changeset	64 }
a61af66fc99e Initial load duke parents: diff changeset	65 return nreplacements;
a61af66fc99e Initial load duke parents: diff changeset	66 }
a61af66fc99e Initial load duke parents: diff changeset	67
a61af66fc99e Initial load duke parents: diff changeset	68 void PhaseMacroExpand::copy_call_debug_info(CallNode oldcall, CallNode newcall) {
a61af66fc99e Initial load duke parents: diff changeset	69 // Copy debug information and adjust JVMState information
a61af66fc99e Initial load duke parents: diff changeset	70 uint old_dbg_start = oldcall->tf()->domain()->cnt();
a61af66fc99e Initial load duke parents: diff changeset	71 uint new_dbg_start = newcall->tf()->domain()->cnt();
a61af66fc99e Initial load duke parents: diff changeset	72 int jvms_adj = new_dbg_start - old_dbg_start;
a61af66fc99e Initial load duke parents: diff changeset	73 assert (new_dbg_start == newcall->req(), "argument count mismatch");
63 eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	74
eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	75 Dict* sosn_map = new Dict(cmpkey,hashkey);
0 a61af66fc99e Initial load duke parents: diff changeset	76 for (uint i = old_dbg_start; i < oldcall->req(); i++) {
63 eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	77 Node* old_in = oldcall->in(i);
eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	78 // Clone old SafePointScalarObjectNodes, adjusting their field contents.
460 424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	79 if (old_in != NULL && old_in->is_SafePointScalarObject()) {
63 eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	80 SafePointScalarObjectNode* old_sosn = old_in->as_SafePointScalarObject();
eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	81 uint old_unique = C->unique();
eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	82 Node* new_in = old_sosn->clone(jvms_adj, sosn_map);
eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	83 if (old_unique != C->unique()) {
601 523ded093c31 6809798: SafePointScalarObject node placed into incorrect block during GCM kvn parents: 584 diff changeset	84 new_in->set_req(0, newcall->in(0)); // reset control edge
63 eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	85 new_in = transform_later(new_in); // Register new node.
eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	86 }
eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	87 old_in = new_in;
eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	88 }
eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	89 newcall->add_req(old_in);
0 a61af66fc99e Initial load duke parents: diff changeset	90 }
63 eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	91
0 a61af66fc99e Initial load duke parents: diff changeset	92 newcall->set_jvms(oldcall->jvms());
a61af66fc99e Initial load duke parents: diff changeset	93 for (JVMState *jvms = newcall->jvms(); jvms != NULL; jvms = jvms->caller()) {
a61af66fc99e Initial load duke parents: diff changeset	94 jvms->set_map(newcall);
a61af66fc99e Initial load duke parents: diff changeset	95 jvms->set_locoff(jvms->locoff()+jvms_adj);
a61af66fc99e Initial load duke parents: diff changeset	96 jvms->set_stkoff(jvms->stkoff()+jvms_adj);
a61af66fc99e Initial load duke parents: diff changeset	97 jvms->set_monoff(jvms->monoff()+jvms_adj);
63 eac007780a58 6671807: (Escape Analysis) Add new ideal node to represent the state of a scalarized object at a safepoint kvn parents: 0 diff changeset	98 jvms->set_scloff(jvms->scloff()+jvms_adj);
0 a61af66fc99e Initial load duke parents: diff changeset	99 jvms->set_endoff(jvms->endoff()+jvms_adj);
a61af66fc99e Initial load duke parents: diff changeset	100 }
a61af66fc99e Initial load duke parents: diff changeset	101 }
a61af66fc99e Initial load duke parents: diff changeset	102
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	103 Node* PhaseMacroExpand::opt_bits_test(Node* ctrl, Node* region, int edge, Node* word, int mask, int bits, bool return_fast_path) {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	104 Node* cmp;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	105 if (mask != 0) {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	106 Node* and_node = transform_later(new (C, 3) AndXNode(word, MakeConX(mask)));
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	107 cmp = transform_later(new (C, 3) CmpXNode(and_node, MakeConX(bits)));
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	108 } else {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	109 cmp = word;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	110 }
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	111 Node* bol = transform_later(new (C, 2) BoolNode(cmp, BoolTest::ne));
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	112 IfNode* iff = new (C, 2) IfNode( ctrl, bol, PROB_MIN, COUNT_UNKNOWN );
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	113 transform_later(iff);
0 a61af66fc99e Initial load duke parents: diff changeset	114
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	115 // Fast path taken.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	116 Node *fast_taken = transform_later( new (C, 1) IfFalseNode(iff) );
0 a61af66fc99e Initial load duke parents: diff changeset	117
a61af66fc99e Initial load duke parents: diff changeset	118 // Fast path not-taken, i.e. slow path
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	119 Node *slow_taken = transform_later( new (C, 1) IfTrueNode(iff) );
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	120
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	121 if (return_fast_path) {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	122 region->init_req(edge, slow_taken); // Capture slow-control
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	123 return fast_taken;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	124 } else {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	125 region->init_req(edge, fast_taken); // Capture fast-control
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	126 return slow_taken;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	127 }
0 a61af66fc99e Initial load duke parents: diff changeset	128 }
a61af66fc99e Initial load duke parents: diff changeset	129
a61af66fc99e Initial load duke parents: diff changeset	130 //--------------------copy_predefined_input_for_runtime_call--------------------
a61af66fc99e Initial load duke parents: diff changeset	131 void PhaseMacroExpand::copy_predefined_input_for_runtime_call(Node * ctrl, CallNode* oldcall, CallNode* call) {
a61af66fc99e Initial load duke parents: diff changeset	132 // Set fixed predefined input arguments
a61af66fc99e Initial load duke parents: diff changeset	133 call->init_req( TypeFunc::Control, ctrl );
a61af66fc99e Initial load duke parents: diff changeset	134 call->init_req( TypeFunc::I_O , oldcall->in( TypeFunc::I_O) );
a61af66fc99e Initial load duke parents: diff changeset	135 call->init_req( TypeFunc::Memory , oldcall->in( TypeFunc::Memory ) ); // ?????
a61af66fc99e Initial load duke parents: diff changeset	136 call->init_req( TypeFunc::ReturnAdr, oldcall->in( TypeFunc::ReturnAdr ) );
a61af66fc99e Initial load duke parents: diff changeset	137 call->init_req( TypeFunc::FramePtr, oldcall->in( TypeFunc::FramePtr ) );
a61af66fc99e Initial load duke parents: diff changeset	138 }
a61af66fc99e Initial load duke parents: diff changeset	139
a61af66fc99e Initial load duke parents: diff changeset	140 //------------------------------make_slow_call---------------------------------
a61af66fc99e Initial load duke parents: diff changeset	141 CallNode* PhaseMacroExpand::make_slow_call(CallNode oldcall, const TypeFunc slow_call_type, address slow_call, const char* leaf_name, Node* slow_path, Node* parm0, Node* parm1) {
a61af66fc99e Initial load duke parents: diff changeset	142
a61af66fc99e Initial load duke parents: diff changeset	143 // Slow-path call
a61af66fc99e Initial load duke parents: diff changeset	144 int size = slow_call_type->domain()->cnt();
a61af66fc99e Initial load duke parents: diff changeset	145 CallNode *call = leaf_name
a61af66fc99e Initial load duke parents: diff changeset	146 ? (CallNode*)new (C, size) CallLeafNode ( slow_call_type, slow_call, leaf_name, TypeRawPtr::BOTTOM )
a61af66fc99e Initial load duke parents: diff changeset	147 : (CallNode*)new (C, size) CallStaticJavaNode( slow_call_type, slow_call, OptoRuntime::stub_name(slow_call), oldcall->jvms()->bci(), TypeRawPtr::BOTTOM );
a61af66fc99e Initial load duke parents: diff changeset	148
a61af66fc99e Initial load duke parents: diff changeset	149 // Slow path call has no side-effects, uses few values
a61af66fc99e Initial load duke parents: diff changeset	150 copy_predefined_input_for_runtime_call(slow_path, oldcall, call );
a61af66fc99e Initial load duke parents: diff changeset	151 if (parm0 != NULL) call->init_req(TypeFunc::Parms+0, parm0);
a61af66fc99e Initial load duke parents: diff changeset	152 if (parm1 != NULL) call->init_req(TypeFunc::Parms+1, parm1);
a61af66fc99e Initial load duke parents: diff changeset	153 copy_call_debug_info(oldcall, call);
a61af66fc99e Initial load duke parents: diff changeset	154 call->set_cnt(PROB_UNLIKELY_MAG(4)); // Same effect as RC_UNCOMMON.
1621 6027dddc26c6 6677629: PhaseIterGVN::subsume_node() should call hash_delete() and add_users_to_worklist() kvn parents: 1609 diff changeset	155 _igvn.replace_node(oldcall, call);
0 a61af66fc99e Initial load duke parents: diff changeset	156 transform_later(call);
a61af66fc99e Initial load duke parents: diff changeset	157
a61af66fc99e Initial load duke parents: diff changeset	158 return call;
a61af66fc99e Initial load duke parents: diff changeset	159 }
a61af66fc99e Initial load duke parents: diff changeset	160
a61af66fc99e Initial load duke parents: diff changeset	161 void PhaseMacroExpand::extract_call_projections(CallNode *call) {
a61af66fc99e Initial load duke parents: diff changeset	162 _fallthroughproj = NULL;
a61af66fc99e Initial load duke parents: diff changeset	163 _fallthroughcatchproj = NULL;
a61af66fc99e Initial load duke parents: diff changeset	164 _ioproj_fallthrough = NULL;
a61af66fc99e Initial load duke parents: diff changeset	165 _ioproj_catchall = NULL;
a61af66fc99e Initial load duke parents: diff changeset	166 _catchallcatchproj = NULL;
a61af66fc99e Initial load duke parents: diff changeset	167 _memproj_fallthrough = NULL;
a61af66fc99e Initial load duke parents: diff changeset	168 _memproj_catchall = NULL;
a61af66fc99e Initial load duke parents: diff changeset	169 _resproj = NULL;
a61af66fc99e Initial load duke parents: diff changeset	170 for (DUIterator_Fast imax, i = call->fast_outs(imax); i < imax; i++) {
a61af66fc99e Initial load duke parents: diff changeset	171 ProjNode *pn = call->fast_out(i)->as_Proj();
a61af66fc99e Initial load duke parents: diff changeset	172 switch (pn->_con) {
a61af66fc99e Initial load duke parents: diff changeset	173 case TypeFunc::Control:
a61af66fc99e Initial load duke parents: diff changeset	174 {
a61af66fc99e Initial load duke parents: diff changeset	175 // For Control (fallthrough) and I_O (catch_all_index) we have CatchProj -> Catch -> Proj
a61af66fc99e Initial load duke parents: diff changeset	176 _fallthroughproj = pn;
a61af66fc99e Initial load duke parents: diff changeset	177 DUIterator_Fast jmax, j = pn->fast_outs(jmax);
a61af66fc99e Initial load duke parents: diff changeset	178 const Node *cn = pn->fast_out(j);
a61af66fc99e Initial load duke parents: diff changeset	179 if (cn->is_Catch()) {
a61af66fc99e Initial load duke parents: diff changeset	180 ProjNode *cpn = NULL;
a61af66fc99e Initial load duke parents: diff changeset	181 for (DUIterator_Fast kmax, k = cn->fast_outs(kmax); k < kmax; k++) {
a61af66fc99e Initial load duke parents: diff changeset	182 cpn = cn->fast_out(k)->as_Proj();
a61af66fc99e Initial load duke parents: diff changeset	183 assert(cpn->is_CatchProj(), "must be a CatchProjNode");
a61af66fc99e Initial load duke parents: diff changeset	184 if (cpn->_con == CatchProjNode::fall_through_index)
a61af66fc99e Initial load duke parents: diff changeset	185 _fallthroughcatchproj = cpn;
a61af66fc99e Initial load duke parents: diff changeset	186 else {
a61af66fc99e Initial load duke parents: diff changeset	187 assert(cpn->_con == CatchProjNode::catch_all_index, "must be correct index.");
a61af66fc99e Initial load duke parents: diff changeset	188 _catchallcatchproj = cpn;
a61af66fc99e Initial load duke parents: diff changeset	189 }
a61af66fc99e Initial load duke parents: diff changeset	190 }
a61af66fc99e Initial load duke parents: diff changeset	191 }
a61af66fc99e Initial load duke parents: diff changeset	192 break;
a61af66fc99e Initial load duke parents: diff changeset	193 }
a61af66fc99e Initial load duke parents: diff changeset	194 case TypeFunc::I_O:
a61af66fc99e Initial load duke parents: diff changeset	195 if (pn->_is_io_use)
a61af66fc99e Initial load duke parents: diff changeset	196 _ioproj_catchall = pn;
a61af66fc99e Initial load duke parents: diff changeset	197 else
a61af66fc99e Initial load duke parents: diff changeset	198 _ioproj_fallthrough = pn;
a61af66fc99e Initial load duke parents: diff changeset	199 break;
a61af66fc99e Initial load duke parents: diff changeset	200 case TypeFunc::Memory:
a61af66fc99e Initial load duke parents: diff changeset	201 if (pn->_is_io_use)
a61af66fc99e Initial load duke parents: diff changeset	202 _memproj_catchall = pn;
a61af66fc99e Initial load duke parents: diff changeset	203 else
a61af66fc99e Initial load duke parents: diff changeset	204 _memproj_fallthrough = pn;
a61af66fc99e Initial load duke parents: diff changeset	205 break;
a61af66fc99e Initial load duke parents: diff changeset	206 case TypeFunc::Parms:
a61af66fc99e Initial load duke parents: diff changeset	207 _resproj = pn;
a61af66fc99e Initial load duke parents: diff changeset	208 break;
a61af66fc99e Initial load duke parents: diff changeset	209 default:
a61af66fc99e Initial load duke parents: diff changeset	210 assert(false, "unexpected projection from allocation node.");
a61af66fc99e Initial load duke parents: diff changeset	211 }
a61af66fc99e Initial load duke parents: diff changeset	212 }
a61af66fc99e Initial load duke parents: diff changeset	213
a61af66fc99e Initial load duke parents: diff changeset	214 }
a61af66fc99e Initial load duke parents: diff changeset	215
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	216 // Eliminate a card mark sequence. p2x is a ConvP2XNode
851 fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	217 void PhaseMacroExpand::eliminate_card_mark(Node* p2x) {
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	218 assert(p2x->Opcode() == Op_CastP2X, "ConvP2XNode required");
851 fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	219 if (!UseG1GC) {
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	220 // vanilla/CMS post barrier
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	221 Node *shift = p2x->unique_out();
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	222 Node *addp = shift->unique_out();
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	223 for (DUIterator_Last jmin, j = addp->last_outs(jmin); j >= jmin; --j) {
3282 149bb459be66 7029167: add support for conditional card marks never parents: 2100 diff changeset	224 Node *mem = addp->last_out(j);
149bb459be66 7029167: add support for conditional card marks never parents: 2100 diff changeset	225 if (UseCondCardMark && mem->is_Load()) {
149bb459be66 7029167: add support for conditional card marks never parents: 2100 diff changeset	226 assert(mem->Opcode() == Op_LoadB, "unexpected code shape");
149bb459be66 7029167: add support for conditional card marks never parents: 2100 diff changeset	227 // The load is checking if the card has been written so
149bb459be66 7029167: add support for conditional card marks never parents: 2100 diff changeset	228 // replace it with zero to fold the test.
149bb459be66 7029167: add support for conditional card marks never parents: 2100 diff changeset	229 _igvn.replace_node(mem, intcon(0));
149bb459be66 7029167: add support for conditional card marks never parents: 2100 diff changeset	230 continue;
149bb459be66 7029167: add support for conditional card marks never parents: 2100 diff changeset	231 }
149bb459be66 7029167: add support for conditional card marks never parents: 2100 diff changeset	232 assert(mem->is_Store(), "store required");
149bb459be66 7029167: add support for conditional card marks never parents: 2100 diff changeset	233 _igvn.replace_node(mem, mem->in(MemNode::Memory));
851 fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	234 }
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	235 } else {
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	236 // G1 pre/post barriers
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	237 assert(p2x->outcnt() == 2, "expects 2 users: Xor and URShift nodes");
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	238 // It could be only one user, URShift node, in Object.clone() instrinsic
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	239 // but the new allocation is passed to arraycopy stub and it could not
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	240 // be scalar replaced. So we don't check the case.
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	241
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	242 // Remove G1 post barrier.
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	243
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	244 // Search for CastP2X->Xor->URShift->Cmp path which
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	245 // checks if the store done to a different from the value's region.
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	246 // And replace Cmp with #0 (false) to collapse G1 post barrier.
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	247 Node* xorx = NULL;
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	248 for (DUIterator_Fast imax, i = p2x->fast_outs(imax); i < imax; i++) {
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	249 Node* u = p2x->fast_out(i);
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	250 if (u->Opcode() == Op_XorX) {
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	251 xorx = u;
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	252 break;
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	253 }
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	254 }
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	255 assert(xorx != NULL, "missing G1 post barrier");
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	256 Node* shift = xorx->unique_out();
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	257 Node* cmpx = shift->unique_out();
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	258 assert(cmpx->is_Cmp() && cmpx->unique_out()->is_Bool() &&
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	259 cmpx->unique_out()->as_Bool()->_test._test == BoolTest::ne,
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	260 "missing region check in G1 post barrier");
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	261 _igvn.replace_node(cmpx, makecon(TypeInt::CC_EQ));
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	262
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	263 // Remove G1 pre barrier.
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	264
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	265 // Search "if (marking != 0)" check and set it to "false".
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	266 Node* this_region = p2x->in(0);
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	267 assert(this_region != NULL, "");
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	268 // There is no G1 pre barrier if previous stored value is NULL
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	269 // (for example, after initialization).
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	270 if (this_region->is_Region() && this_region->req() == 3) {
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	271 int ind = 1;
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	272 if (!this_region->in(ind)->is_IfFalse()) {
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	273 ind = 2;
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	274 }
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	275 if (this_region->in(ind)->is_IfFalse()) {
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	276 Node* bol = this_region->in(ind)->in(0)->in(1);
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	277 assert(bol->is_Bool(), "");
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	278 cmpx = bol->in(1);
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	279 if (bol->as_Bool()->_test._test == BoolTest::ne &&
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	280 cmpx->is_Cmp() && cmpx->in(2) == intcon(0) &&
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	281 cmpx->in(1)->is_Load()) {
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	282 Node* adr = cmpx->in(1)->as_Load()->in(MemNode::Address);
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	283 const int marking_offset = in_bytes(JavaThread::satb_mark_queue_offset() +
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	284 PtrQueue::byte_offset_of_active());
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	285 if (adr->is_AddP() && adr->in(AddPNode::Base) == top() &&
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	286 adr->in(AddPNode::Address)->Opcode() == Op_ThreadLocal &&
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	287 adr->in(AddPNode::Offset) == MakeConX(marking_offset)) {
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	288 _igvn.replace_node(cmpx, makecon(TypeInt::CC_EQ));
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	289 }
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	290 }
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	291 }
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	292 }
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	293 // Now CastP2X can be removed since it is used only on dead path
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	294 // which currently still alive until igvn optimize it.
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	295 assert(p2x->unique_out()->Opcode() == Op_URShiftX, "");
fc4be448891f 6851742: (EA) allocation elimination doesn't work with UseG1GC kvn parents: 780 diff changeset	296 _igvn.replace_node(p2x, top());
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	297 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	298 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	299
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	300 // Search for a memory operation for the specified memory slice.
253 b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	301 static Node scan_mem_chain(Node mem, int alias_idx, int offset, Node start_mem, Node alloc, PhaseGVN *phase) {
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	302 Node *orig_mem = mem;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	303 Node *alloc_mem = alloc->in(TypeFunc::Memory);
253 b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	304 const TypeOopPtr *tinst = phase->C->get_adr_type(alias_idx)->isa_oopptr();
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	305 while (true) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	306 if (mem == alloc_mem \|\| mem == start_mem ) {
605 98cb887364d3 6810672: Comment typos twisti parents: 601 diff changeset	307 return mem; // hit one of our sentinels
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	308 } else if (mem->is_MergeMem()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	309 mem = mem->as_MergeMem()->memory_at(alias_idx);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	310 } else if (mem->is_Proj() && mem->as_Proj()->_con == TypeFunc::Memory) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	311 Node *in = mem->in(0);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	312 // we can safely skip over safepoints, calls, locks and membars because we
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	313 // already know that the object is safe to eliminate.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	314 if (in->is_Initialize() && in->as_Initialize()->allocation() == alloc) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	315 return in;
253 b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	316 } else if (in->is_Call()) {
b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	317 CallNode *call = in->as_Call();
b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	318 if (!call->may_modify(tinst, phase)) {
b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	319 mem = call->in(TypeFunc::Memory);
b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	320 }
b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	321 mem = in->in(TypeFunc::Memory);
b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	322 } else if (in->is_MemBar()) {
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	323 mem = in->in(TypeFunc::Memory);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	324 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	325 assert(false, "unexpected projection");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	326 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	327 } else if (mem->is_Store()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	328 const TypePtr* atype = mem->as_Store()->adr_type();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	329 int adr_idx = Compile::current()->get_alias_index(atype);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	330 if (adr_idx == alias_idx) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	331 assert(atype->isa_oopptr(), "address type must be oopptr");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	332 int adr_offset = atype->offset();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	333 uint adr_iid = atype->is_oopptr()->instance_id();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	334 // Array elements references have the same alias_idx
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	335 // but different offset and different instance_id.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	336 if (adr_offset == offset && adr_iid == alloc->_idx)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	337 return mem;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	338 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	339 assert(adr_idx == Compile::AliasIdxRaw, "address must match or be raw");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	340 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	341 mem = mem->in(MemNode::Memory);
1100 f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	342 } else if (mem->is_ClearArray()) {
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	343 if (!ClearArrayNode::step_through(&mem, alloc->_idx, phase)) {
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	344 // Can not bypass initialization of the instance
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	345 // we are looking.
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	346 debug_only(intptr_t offset;)
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	347 assert(alloc == AllocateNode::Ideal_allocation(mem->in(3), phase, offset), "sanity");
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	348 InitializeNode* init = alloc->as_Allocate()->initialization();
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	349 // We are looking for stored value, return Initialize node
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	350 // or memory edge from Allocate node.
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	351 if (init != NULL)
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	352 return init;
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	353 else
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	354 return alloc->in(TypeFunc::Memory); // It will produce zero value (see callers).
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	355 }
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	356 // Otherwise skip it (the call updated 'mem' value).
584 49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	357 } else if (mem->Opcode() == Op_SCMemProj) {
49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	358 assert(mem->in(0)->is_LoadStore(), "sanity");
49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	359 const TypePtr* atype = mem->in(0)->in(MemNode::Address)->bottom_type()->is_ptr();
49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	360 int adr_idx = Compile::current()->get_alias_index(atype);
49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	361 if (adr_idx == alias_idx) {
49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	362 assert(false, "Object is not scalar replaceable if a LoadStore node access its field");
49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	363 return NULL;
49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	364 }
49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	365 mem = mem->in(0)->in(MemNode::Memory);
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	366 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	367 return mem;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	368 }
247 02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	369 assert(mem != orig_mem, "dead memory loop");
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	370 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	371 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	372
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	373 //
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	374 // Given a Memory Phi, compute a value Phi containing the values from stores
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	375 // on the input paths.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	376 // Note: this function is recursive, its depth is limied by the "level" argument
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	377 // Returns the computed Phi, or NULL if it cannot compute it.
247 02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	378 Node PhaseMacroExpand::value_from_mem_phi(Node mem, BasicType ft, const Type phi_type, const TypeOopPtr adr_t, Node alloc, Node_Stack value_phis, int level) {
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	379 assert(mem->is_Phi(), "sanity");
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	380 int alias_idx = C->get_alias_index(adr_t);
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	381 int offset = adr_t->offset();
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	382 int instance_id = adr_t->instance_id();
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	383
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	384 // Check if an appropriate value phi already exists.
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	385 Node* region = mem->in(0);
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	386 for (DUIterator_Fast kmax, k = region->fast_outs(kmax); k < kmax; k++) {
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	387 Node* phi = region->fast_out(k);
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	388 if (phi->is_Phi() && phi != mem &&
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	389 phi->as_Phi()->is_same_inst_field(phi_type, instance_id, alias_idx, offset)) {
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	390 return phi;
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	391 }
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	392 }
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	393 // Check if an appropriate new value phi already exists.
3788 e3cbc9ddd434 7044738: Loop unroll optimization causes incorrect result kvn parents: 3754 diff changeset	394 Node* new_phi = value_phis->find(mem->_idx);
e3cbc9ddd434 7044738: Loop unroll optimization causes incorrect result kvn parents: 3754 diff changeset	395 if (new_phi != NULL)
e3cbc9ddd434 7044738: Loop unroll optimization causes incorrect result kvn parents: 3754 diff changeset	396 return new_phi;
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	397
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	398 if (level <= 0) {
253 b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	399 return NULL; // Give up: phi tree too deep
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	400 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	401 Node *start_mem = C->start()->proj_out(TypeFunc::Memory);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	402 Node *alloc_mem = alloc->in(TypeFunc::Memory);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	403
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	404 uint length = mem->req();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	405 GrowableArray <Node *> values(length, length, NULL);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	406
247 02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	407 // create a new Phi for the value
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	408 PhiNode *phi = new (C, length) PhiNode(mem->in(0), phi_type, NULL, instance_id, alias_idx, offset);
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	409 transform_later(phi);
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	410 value_phis->push(phi, mem->_idx);
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	411
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	412 for (uint j = 1; j < length; j++) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	413 Node *in = mem->in(j);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	414 if (in == NULL \|\| in->is_top()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	415 values.at_put(j, in);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	416 } else {
253 b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	417 Node *val = scan_mem_chain(in, alias_idx, offset, start_mem, alloc, &_igvn);
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	418 if (val == start_mem \|\| val == alloc_mem) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	419 // hit a sentinel, return appropriate 0 value
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	420 values.at_put(j, _igvn.zerocon(ft));
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	421 continue;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	422 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	423 if (val->is_Initialize()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	424 val = val->as_Initialize()->find_captured_store(offset, type2aelembytes(ft), &_igvn);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	425 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	426 if (val == NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	427 return NULL; // can't find a value on this path
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	428 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	429 if (val == mem) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	430 values.at_put(j, mem);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	431 } else if (val->is_Store()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	432 values.at_put(j, val->in(MemNode::ValueIn));
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	433 } else if(val->is_Proj() && val->in(0) == alloc) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	434 values.at_put(j, _igvn.zerocon(ft));
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	435 } else if (val->is_Phi()) {
247 02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	436 val = value_from_mem_phi(val, ft, phi_type, adr_t, alloc, value_phis, level-1);
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	437 if (val == NULL) {
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	438 return NULL;
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	439 }
247 02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	440 values.at_put(j, val);
584 49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	441 } else if (val->Opcode() == Op_SCMemProj) {
49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	442 assert(val->in(0)->is_LoadStore(), "sanity");
49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	443 assert(false, "Object is not scalar replaceable if a LoadStore node access its field");
49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	444 return NULL;
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	445 } else {
584 49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	446 #ifdef ASSERT
49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	447 val->dump();
253 b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	448 assert(false, "unknown node on this path");
584 49a36a80b0c7 6802499: EA: assert(false,"unknown node on this path") kvn parents: 565 diff changeset	449 #endif
253 b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	450 return NULL; // unknown node on this path
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	451 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	452 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	453 }
247 02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	454 // Set Phi's inputs
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	455 for (uint j = 1; j < length; j++) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	456 if (values.at(j) == mem) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	457 phi->init_req(j, phi);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	458 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	459 phi->init_req(j, values.at(j));
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	460 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	461 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	462 return phi;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	463 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	464
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	465 // Search the last value stored into the object's field.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	466 Node PhaseMacroExpand::value_from_mem(Node sfpt_mem, BasicType ft, const Type ftype, const TypeOopPtr adr_t, Node *alloc) {
223 1dd146f17531 6716441: error in meet with +DoEscapeAnalysis kvn parents: 221 diff changeset	467 assert(adr_t->is_known_instance_field(), "instance required");
1dd146f17531 6716441: error in meet with +DoEscapeAnalysis kvn parents: 221 diff changeset	468 int instance_id = adr_t->instance_id();
1dd146f17531 6716441: error in meet with +DoEscapeAnalysis kvn parents: 221 diff changeset	469 assert((uint)instance_id == alloc->_idx, "wrong allocation");
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	470
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	471 int alias_idx = C->get_alias_index(adr_t);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	472 int offset = adr_t->offset();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	473 Node *start_mem = C->start()->proj_out(TypeFunc::Memory);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	474 Node *alloc_ctrl = alloc->in(TypeFunc::Control);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	475 Node *alloc_mem = alloc->in(TypeFunc::Memory);
247 02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	476 Arena *a = Thread::current()->resource_area();
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	477 VectorSet visited(a);
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	478
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	479
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	480 bool done = sfpt_mem == alloc_mem;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	481 Node *mem = sfpt_mem;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	482 while (!done) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	483 if (visited.test_set(mem->_idx)) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	484 return NULL; // found a loop, give up
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	485 }
253 b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	486 mem = scan_mem_chain(mem, alias_idx, offset, start_mem, alloc, &_igvn);
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	487 if (mem == start_mem \|\| mem == alloc_mem) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	488 done = true; // hit a sentinel, return appropriate 0 value
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	489 } else if (mem->is_Initialize()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	490 mem = mem->as_Initialize()->find_captured_store(offset, type2aelembytes(ft), &_igvn);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	491 if (mem == NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	492 done = true; // Something go wrong.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	493 } else if (mem->is_Store()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	494 const TypePtr* atype = mem->as_Store()->adr_type();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	495 assert(C->get_alias_index(atype) == Compile::AliasIdxRaw, "store is correct memory slice");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	496 done = true;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	497 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	498 } else if (mem->is_Store()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	499 const TypeOopPtr* atype = mem->as_Store()->adr_type()->isa_oopptr();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	500 assert(atype != NULL, "address type must be oopptr");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	501 assert(C->get_alias_index(atype) == alias_idx &&
223 1dd146f17531 6716441: error in meet with +DoEscapeAnalysis kvn parents: 221 diff changeset	502 atype->is_known_instance_field() && atype->offset() == offset &&
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	503 atype->instance_id() == instance_id, "store is correct memory slice");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	504 done = true;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	505 } else if (mem->is_Phi()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	506 // try to find a phi's unique input
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	507 Node *unique_input = NULL;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	508 Node *top = C->top();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	509 for (uint i = 1; i < mem->req(); i++) {
253 b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	510 Node *n = scan_mem_chain(mem->in(i), alias_idx, offset, start_mem, alloc, &_igvn);
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	511 if (n == NULL \|\| n == top \|\| n == mem) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	512 continue;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	513 } else if (unique_input == NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	514 unique_input = n;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	515 } else if (unique_input != n) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	516 unique_input = top;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	517 break;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	518 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	519 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	520 if (unique_input != NULL && unique_input != top) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	521 mem = unique_input;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	522 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	523 done = true;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	524 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	525 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	526 assert(false, "unexpected node");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	527 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	528 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	529 if (mem != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	530 if (mem == start_mem \|\| mem == alloc_mem) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	531 // hit a sentinel, return appropriate 0 value
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	532 return _igvn.zerocon(ft);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	533 } else if (mem->is_Store()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	534 return mem->in(MemNode::ValueIn);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	535 } else if (mem->is_Phi()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	536 // attempt to produce a Phi reflecting the values on the input paths of the Phi
247 02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	537 Node_Stack value_phis(a, 8);
253 b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	538 Node * phi = value_from_mem_phi(mem, ft, ftype, adr_t, alloc, &value_phis, ValueSearchLimit);
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	539 if (phi != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	540 return phi;
247 02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	541 } else {
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	542 // Kill all new Phis
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	543 while(value_phis.is_nonempty()) {
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	544 Node* n = value_phis.node();
1621 6027dddc26c6 6677629: PhaseIterGVN::subsume_node() should call hash_delete() and add_users_to_worklist() kvn parents: 1609 diff changeset	545 _igvn.replace_node(n, C->top());
247 02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	546 value_phis.pop();
02a35ad4adf8 6723160: Nightly failure: Error: meet not symmetric kvn parents: 235 diff changeset	547 }
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	548 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	549 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	550 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	551 // Something go wrong.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	552 return NULL;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	553 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	554
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	555 // Check the possibility of scalar replacement.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	556 bool PhaseMacroExpand::can_eliminate_allocation(AllocateNode alloc, GrowableArray <SafePointNode >& safepoints) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	557 // Scan the uses of the allocation to check for anything that would
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	558 // prevent us from eliminating it.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	559 NOT_PRODUCT( const char* fail_eliminate = NULL; )
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	560 DEBUG_ONLY( Node* disq_node = NULL; )
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	561 bool can_eliminate = true;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	562
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	563 Node* res = alloc->result_cast();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	564 const TypeOopPtr* res_type = NULL;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	565 if (res == NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	566 // All users were eliminated.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	567 } else if (!res->is_CheckCastPP()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	568 alloc->_is_scalar_replaceable = false; // don't try again
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	569 NOT_PRODUCT(fail_eliminate = "Allocation does not have unique CheckCastPP";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	570 can_eliminate = false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	571 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	572 res_type = _igvn.type(res)->isa_oopptr();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	573 if (res_type == NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	574 NOT_PRODUCT(fail_eliminate = "Neither instance or array allocation";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	575 can_eliminate = false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	576 } else if (res_type->isa_aryptr()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	577 int length = alloc->in(AllocateNode::ALength)->find_int_con(-1);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	578 if (length < 0) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	579 NOT_PRODUCT(fail_eliminate = "Array's size is not constant";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	580 can_eliminate = false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	581 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	582 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	583 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	584
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	585 if (can_eliminate && res != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	586 for (DUIterator_Fast jmax, j = res->fast_outs(jmax);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	587 j < jmax && can_eliminate; j++) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	588 Node* use = res->fast_out(j);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	589
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	590 if (use->is_AddP()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	591 const TypePtr* addp_type = _igvn.type(use)->is_ptr();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	592 int offset = addp_type->offset();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	593
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	594 if (offset == Type::OffsetTop \|\| offset == Type::OffsetBot) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	595 NOT_PRODUCT(fail_eliminate = "Undefined field referrence";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	596 can_eliminate = false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	597 break;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	598 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	599 for (DUIterator_Fast kmax, k = use->fast_outs(kmax);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	600 k < kmax && can_eliminate; k++) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	601 Node* n = use->fast_out(k);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	602 if (!n->is_Store() && n->Opcode() != Op_CastP2X) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	603 DEBUG_ONLY(disq_node = n;)
253 b0fe4deeb9fb 6726999: nsk/stress/jck12a/jck12a010 assert(n != null,"Bad immediate dominator info.") kvn parents: 247 diff changeset	604 if (n->is_Load() \|\| n->is_LoadStore()) {
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	605 NOT_PRODUCT(fail_eliminate = "Field load";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	606 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	607 NOT_PRODUCT(fail_eliminate = "Not store field referrence";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	608 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	609 can_eliminate = false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	610 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	611 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	612 } else if (use->is_SafePoint()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	613 SafePointNode* sfpt = use->as_SafePoint();
168 7793bd37a336 6705887: Compressed Oops: generate x64 addressing and implicit null checks with narrow oops kvn parents: 163 diff changeset	614 if (sfpt->is_Call() && sfpt->as_Call()->has_non_debug_use(res)) {
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	615 // Object is passed as argument.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	616 DEBUG_ONLY(disq_node = use;)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	617 NOT_PRODUCT(fail_eliminate = "Object is passed as argument";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	618 can_eliminate = false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	619 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	620 Node* sfptMem = sfpt->memory();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	621 if (sfptMem == NULL \|\| sfptMem->is_top()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	622 DEBUG_ONLY(disq_node = use;)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	623 NOT_PRODUCT(fail_eliminate = "NULL or TOP memory";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	624 can_eliminate = false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	625 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	626 safepoints.append_if_missing(sfpt);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	627 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	628 } else if (use->Opcode() != Op_CastP2X) { // CastP2X is used by card mark
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	629 if (use->is_Phi()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	630 if (use->outcnt() == 1 && use->unique_out()->Opcode() == Op_Return) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	631 NOT_PRODUCT(fail_eliminate = "Object is return value";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	632 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	633 NOT_PRODUCT(fail_eliminate = "Object is referenced by Phi";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	634 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	635 DEBUG_ONLY(disq_node = use;)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	636 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	637 if (use->Opcode() == Op_Return) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	638 NOT_PRODUCT(fail_eliminate = "Object is return value";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	639 }else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	640 NOT_PRODUCT(fail_eliminate = "Object is referenced by node";)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	641 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	642 DEBUG_ONLY(disq_node = use;)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	643 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	644 can_eliminate = false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	645 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	646 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	647 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	648
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	649 #ifndef PRODUCT
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	650 if (PrintEliminateAllocations) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	651 if (can_eliminate) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	652 tty->print("Scalar ");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	653 if (res == NULL)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	654 alloc->dump();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	655 else
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	656 res->dump();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	657 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	658 tty->print("NotScalar (%s)", fail_eliminate);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	659 if (res == NULL)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	660 alloc->dump();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	661 else
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	662 res->dump();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	663 #ifdef ASSERT
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	664 if (disq_node != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	665 tty->print(" >>>> ");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	666 disq_node->dump();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	667 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	668 #endif /ASSERT/
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	669 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	670 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	671 #endif
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	672 return can_eliminate;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	673 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	674
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	675 // Do scalar replacement.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	676 bool PhaseMacroExpand::scalar_replacement(AllocateNode alloc, GrowableArray <SafePointNode >& safepoints) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	677 GrowableArray <SafePointNode *> safepoints_done;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	678
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	679 ciKlass* klass = NULL;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	680 ciInstanceKlass* iklass = NULL;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	681 int nfields = 0;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	682 int array_base;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	683 int element_size;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	684 BasicType basic_elem_type;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	685 ciType* elem_type;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	686
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	687 Node* res = alloc->result_cast();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	688 const TypeOopPtr* res_type = NULL;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	689 if (res != NULL) { // Could be NULL when there are no users
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	690 res_type = _igvn.type(res)->isa_oopptr();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	691 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	692
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	693 if (res != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	694 klass = res_type->klass();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	695 if (res_type->isa_instptr()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	696 // find the fields of the class which will be needed for safepoint debug information
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	697 assert(klass->is_instance_klass(), "must be an instance klass.");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	698 iklass = klass->as_instance_klass();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	699 nfields = iklass->nof_nonstatic_fields();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	700 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	701 // find the array's elements which will be needed for safepoint debug information
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	702 nfields = alloc->in(AllocateNode::ALength)->find_int_con(-1);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	703 assert(klass->is_array_klass() && nfields >= 0, "must be an array klass.");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	704 elem_type = klass->as_array_klass()->element_type();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	705 basic_elem_type = elem_type->basic_type();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	706 array_base = arrayOopDesc::base_offset_in_bytes(basic_elem_type);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	707 element_size = type2aelembytes(basic_elem_type);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	708 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	709 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	710 //
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	711 // Process the safepoint uses
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	712 //
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	713 while (safepoints.length() > 0) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	714 SafePointNode* sfpt = safepoints.pop();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	715 Node* mem = sfpt->memory();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	716 uint first_ind = sfpt->req();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	717 SafePointScalarObjectNode* sobj = new (C, 1) SafePointScalarObjectNode(res_type,
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	718 #ifdef ASSERT
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	719 alloc,
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	720 #endif
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	721 first_ind, nfields);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	722 sobj->init_req(0, sfpt->in(TypeFunc::Control));
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	723 transform_later(sobj);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	724
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	725 // Scan object's fields adding an input to the safepoint for each field.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	726 for (int j = 0; j < nfields; j++) {
306 af945ba2e739 6741738: TypePtr::add_offset() set incorrect offset when the add overflows kvn parents: 253 diff changeset	727 intptr_t offset;
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	728 ciField* field = NULL;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	729 if (iklass != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	730 field = iklass->nonstatic_field_at(j);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	731 offset = field->offset();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	732 elem_type = field->type();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	733 basic_elem_type = field->layout_type();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	734 } else {
306 af945ba2e739 6741738: TypePtr::add_offset() set incorrect offset when the add overflows kvn parents: 253 diff changeset	735 offset = array_base + j * (intptr_t)element_size;
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	736 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	737
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	738 const Type *field_type;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	739 // The next code is taken from Parse::do_get_xxx().
124 b130b98db9cf 6689060: Escape Analysis does not work with Compressed Oops kvn parents: 113 diff changeset	740 if (basic_elem_type == T_OBJECT \|\| basic_elem_type == T_ARRAY) {
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	741 if (!elem_type->is_loaded()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	742 field_type = TypeInstPtr::BOTTOM;
1682 e5dfb3ccb88b 6969569: assert(is_static() && is_constant()) failed: illegal call to constant_value() kvn parents: 1621 diff changeset	743 } else if (field != NULL && field->is_constant() && field->is_static()) {
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	744 // This can happen if the constant oop is non-perm.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	745 ciObject* con = field->constant_value().as_object();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	746 // Do not "join" in the previous type; it doesn't add value,
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	747 // and may yield a vacuous result if the field is of interface type.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	748 field_type = TypeOopPtr::make_from_constant(con)->isa_oopptr();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	749 assert(field_type != NULL, "field singleton type must be consistent");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	750 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	751 field_type = TypeOopPtr::make_from_klass(elem_type->as_klass());
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	752 }
124 b130b98db9cf 6689060: Escape Analysis does not work with Compressed Oops kvn parents: 113 diff changeset	753 if (UseCompressedOops) {
221 1e026f8da827 6710487: More than half of JDI Regression tests hang with COOPs in -Xcomp mode kvn parents: 216 diff changeset	754 field_type = field_type->make_narrowoop();
124 b130b98db9cf 6689060: Escape Analysis does not work with Compressed Oops kvn parents: 113 diff changeset	755 basic_elem_type = T_NARROWOOP;
b130b98db9cf 6689060: Escape Analysis does not work with Compressed Oops kvn parents: 113 diff changeset	756 }
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	757 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	758 field_type = Type::get_const_basic_type(basic_elem_type);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	759 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	760
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	761 const TypeOopPtr *field_addr_type = res_type->add_offset(offset)->isa_oopptr();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	762
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	763 Node *field_val = value_from_mem(mem, basic_elem_type, field_type, field_addr_type, alloc);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	764 if (field_val == NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	765 // we weren't able to find a value for this field,
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	766 // give up on eliminating this allocation
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	767 alloc->_is_scalar_replaceable = false; // don't try again
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	768 // remove any extra entries we added to the safepoint
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	769 uint last = sfpt->req() - 1;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	770 for (int k = 0; k < j; k++) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	771 sfpt->del_req(last--);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	772 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	773 // rollback processed safepoints
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	774 while (safepoints_done.length() > 0) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	775 SafePointNode* sfpt_done = safepoints_done.pop();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	776 // remove any extra entries we added to the safepoint
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	777 last = sfpt_done->req() - 1;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	778 for (int k = 0; k < nfields; k++) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	779 sfpt_done->del_req(last--);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	780 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	781 JVMState *jvms = sfpt_done->jvms();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	782 jvms->set_endoff(sfpt_done->req());
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	783 // Now make a pass over the debug information replacing any references
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	784 // to SafePointScalarObjectNode with the allocated object.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	785 int start = jvms->debug_start();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	786 int end = jvms->debug_end();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	787 for (int i = start; i < end; i++) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	788 if (sfpt_done->in(i)->is_SafePointScalarObject()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	789 SafePointScalarObjectNode* scobj = sfpt_done->in(i)->as_SafePointScalarObject();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	790 if (scobj->first_index() == sfpt_done->req() &&
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	791 scobj->n_fields() == (uint)nfields) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	792 assert(scobj->alloc() == alloc, "sanity");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	793 sfpt_done->set_req(i, res);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	794 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	795 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	796 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	797 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	798 #ifndef PRODUCT
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	799 if (PrintEliminateAllocations) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	800 if (field != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	801 tty->print("=== At SafePoint node %d can't find value of Field: ",
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	802 sfpt->_idx);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	803 field->print();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	804 int field_idx = C->get_alias_index(field_addr_type);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	805 tty->print(" (alias_idx=%d)", field_idx);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	806 } else { // Array's element
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	807 tty->print("=== At SafePoint node %d can't find value of array element [%d]",
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	808 sfpt->_idx, j);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	809 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	810 tty->print(", which prevents elimination of: ");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	811 if (res == NULL)
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	812 alloc->dump();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	813 else
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	814 res->dump();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	815 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	816 #endif
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	817 return false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	818 }
124 b130b98db9cf 6689060: Escape Analysis does not work with Compressed Oops kvn parents: 113 diff changeset	819 if (UseCompressedOops && field_type->isa_narrowoop()) {
b130b98db9cf 6689060: Escape Analysis does not work with Compressed Oops kvn parents: 113 diff changeset	820 // Enable "DecodeN(EncodeP(Allocate)) --> Allocate" transformation
b130b98db9cf 6689060: Escape Analysis does not work with Compressed Oops kvn parents: 113 diff changeset	821 // to be able scalar replace the allocation.
221 1e026f8da827 6710487: More than half of JDI Regression tests hang with COOPs in -Xcomp mode kvn parents: 216 diff changeset	822 if (field_val->is_EncodeP()) {
1e026f8da827 6710487: More than half of JDI Regression tests hang with COOPs in -Xcomp mode kvn parents: 216 diff changeset	823 field_val = field_val->in(1);
1e026f8da827 6710487: More than half of JDI Regression tests hang with COOPs in -Xcomp mode kvn parents: 216 diff changeset	824 } else {
1e026f8da827 6710487: More than half of JDI Regression tests hang with COOPs in -Xcomp mode kvn parents: 216 diff changeset	825 field_val = transform_later(new (C, 2) DecodeNNode(field_val, field_val->bottom_type()->make_ptr()));
1e026f8da827 6710487: More than half of JDI Regression tests hang with COOPs in -Xcomp mode kvn parents: 216 diff changeset	826 }
124 b130b98db9cf 6689060: Escape Analysis does not work with Compressed Oops kvn parents: 113 diff changeset	827 }
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	828 sfpt->add_req(field_val);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	829 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	830 JVMState *jvms = sfpt->jvms();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	831 jvms->set_endoff(sfpt->req());
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	832 // Now make a pass over the debug information replacing any references
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	833 // to the allocated object with "sobj"
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	834 int start = jvms->debug_start();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	835 int end = jvms->debug_end();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	836 for (int i = start; i < end; i++) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	837 if (sfpt->in(i) == res) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	838 sfpt->set_req(i, sobj);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	839 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	840 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	841 safepoints_done.append_if_missing(sfpt); // keep it for rollback
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	842 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	843 return true;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	844 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	845
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	846 // Process users of eliminated allocation.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	847 void PhaseMacroExpand::process_users_of_allocation(AllocateNode *alloc) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	848 Node* res = alloc->result_cast();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	849 if (res != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	850 for (DUIterator_Last jmin, j = res->last_outs(jmin); j >= jmin; ) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	851 Node *use = res->last_out(j);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	852 uint oc1 = res->outcnt();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	853
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	854 if (use->is_AddP()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	855 for (DUIterator_Last kmin, k = use->last_outs(kmin); k >= kmin; ) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	856 Node *n = use->last_out(k);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	857 uint oc2 = use->outcnt();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	858 if (n->is_Store()) {
1100 f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	859 #ifdef ASSERT
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	860 // Verify that there is no dependent MemBarVolatile nodes,
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	861 // they should be removed during IGVN, see MemBarNode::Ideal().
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	862 for (DUIterator_Fast pmax, p = n->fast_outs(pmax);
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	863 p < pmax; p++) {
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	864 Node* mb = n->fast_out(p);
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	865 assert(mb->is_Initialize() \|\| !mb->is_MemBar() \|\|
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	866 mb->req() <= MemBarNode::Precedent \|\|
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	867 mb->in(MemBarNode::Precedent) != n,
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	868 "MemBarVolatile should be eliminated for non-escaping object");
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	869 }
f96a1a986f7b 6895383: JCK test throws NPE for method compiled with Escape Analysis kvn parents: 1080 diff changeset	870 #endif
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	871 _igvn.replace_node(n, n->in(MemNode::Memory));
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	872 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	873 eliminate_card_mark(n);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	874 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	875 k -= (oc2 - use->outcnt());
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	876 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	877 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	878 eliminate_card_mark(use);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	879 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	880 j -= (oc1 - res->outcnt());
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	881 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	882 assert(res->outcnt() == 0, "all uses of allocated objects must be deleted");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	883 _igvn.remove_dead_node(res);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	884 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	885
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	886 //
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	887 // Process other users of allocation's projections
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	888 //
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	889 if (_resproj != NULL && _resproj->outcnt() != 0) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	890 for (DUIterator_Last jmin, j = _resproj->last_outs(jmin); j >= jmin; ) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	891 Node *use = _resproj->last_out(j);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	892 uint oc1 = _resproj->outcnt();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	893 if (use->is_Initialize()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	894 // Eliminate Initialize node.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	895 InitializeNode *init = use->as_Initialize();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	896 assert(init->outcnt() <= 2, "only a control and memory projection expected");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	897 Node *ctrl_proj = init->proj_out(TypeFunc::Control);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	898 if (ctrl_proj != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	899 assert(init->in(TypeFunc::Control) == _fallthroughcatchproj, "allocation control projection");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	900 _igvn.replace_node(ctrl_proj, _fallthroughcatchproj);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	901 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	902 Node *mem_proj = init->proj_out(TypeFunc::Memory);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	903 if (mem_proj != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	904 Node *mem = init->in(TypeFunc::Memory);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	905 #ifdef ASSERT
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	906 if (mem->is_MergeMem()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	907 assert(mem->in(TypeFunc::Memory) == _memproj_fallthrough, "allocation memory projection");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	908 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	909 assert(mem == _memproj_fallthrough, "allocation memory projection");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	910 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	911 #endif
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	912 _igvn.replace_node(mem_proj, mem);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	913 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	914 } else if (use->is_AddP()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	915 // raw memory addresses used only by the initialization
708 f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	916 _igvn.replace_node(use, C->top());
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	917 } else {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	918 assert(false, "only Initialize or AddP expected");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	919 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	920 j -= (oc1 - _resproj->outcnt());
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	921 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	922 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	923 if (_fallthroughcatchproj != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	924 _igvn.replace_node(_fallthroughcatchproj, alloc->in(TypeFunc::Control));
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	925 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	926 if (_memproj_fallthrough != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	927 _igvn.replace_node(_memproj_fallthrough, alloc->in(TypeFunc::Memory));
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	928 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	929 if (_memproj_catchall != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	930 _igvn.replace_node(_memproj_catchall, C->top());
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	931 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	932 if (_ioproj_fallthrough != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	933 _igvn.replace_node(_ioproj_fallthrough, alloc->in(TypeFunc::I_O));
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	934 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	935 if (_ioproj_catchall != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	936 _igvn.replace_node(_ioproj_catchall, C->top());
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	937 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	938 if (_catchallcatchproj != NULL) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	939 _igvn.replace_node(_catchallcatchproj, C->top());
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	940 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	941 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	942
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	943 bool PhaseMacroExpand::eliminate_allocate_node(AllocateNode *alloc) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	944
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	945 if (!EliminateAllocations \|\| !alloc->_is_scalar_replaceable) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	946 return false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	947 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	948
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	949 extract_call_projections(alloc);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	950
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	951 GrowableArray <SafePointNode *> safepoints;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	952 if (!can_eliminate_allocation(alloc, safepoints)) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	953 return false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	954 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	955
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	956 if (!scalar_replacement(alloc, safepoints)) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	957 return false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	958 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	959
1080 7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	960 CompileLog* log = C->log();
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	961 if (log != NULL) {
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	962 Node* klass = alloc->in(AllocateNode::KlassNode);
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	963 const TypeKlassPtr* tklass = _igvn.type(klass)->is_klassptr();
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	964 log->head("eliminate_allocation type='%d'",
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	965 log->identify(tklass->klass()));
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	966 JVMState* p = alloc->jvms();
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	967 while (p != NULL) {
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	968 log->elem("jvms bci='%d' method='%d'", p->bci(), log->identify(p->method()));
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	969 p = p->caller();
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	970 }
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	971 log->tail("eliminate_allocation");
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	972 }
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	973
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	974 process_users_of_allocation(alloc);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	975
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	976 #ifndef PRODUCT
1080 7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	977 if (PrintEliminateAllocations) {
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	978 if (alloc->is_AllocateArray())
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	979 tty->print_cr("++++ Eliminated: %d AllocateArray", alloc->_idx);
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	980 else
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	981 tty->print_cr("++++ Eliminated: %d Allocate", alloc->_idx);
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	982 }
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	983 #endif
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	984
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	985 return true;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	986 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	987
0 a61af66fc99e Initial load duke parents: diff changeset	988
a61af66fc99e Initial load duke parents: diff changeset	989 //---------------------------set_eden_pointers-------------------------
a61af66fc99e Initial load duke parents: diff changeset	990 void PhaseMacroExpand::set_eden_pointers(Node* &eden_top_adr, Node* &eden_end_adr) {
a61af66fc99e Initial load duke parents: diff changeset	991 if (UseTLAB) { // Private allocation: load from TLS
a61af66fc99e Initial load duke parents: diff changeset	992 Node* thread = transform_later(new (C, 1) ThreadLocalNode());
a61af66fc99e Initial load duke parents: diff changeset	993 int tlab_top_offset = in_bytes(JavaThread::tlab_top_offset());
a61af66fc99e Initial load duke parents: diff changeset	994 int tlab_end_offset = in_bytes(JavaThread::tlab_end_offset());
a61af66fc99e Initial load duke parents: diff changeset	995 eden_top_adr = basic_plus_adr(top()/not oop/, thread, tlab_top_offset);
a61af66fc99e Initial load duke parents: diff changeset	996 eden_end_adr = basic_plus_adr(top()/not oop/, thread, tlab_end_offset);
a61af66fc99e Initial load duke parents: diff changeset	997 } else { // Shared allocation: load from globals
a61af66fc99e Initial load duke parents: diff changeset	998 CollectedHeap* ch = Universe::heap();
a61af66fc99e Initial load duke parents: diff changeset	999 address top_adr = (address)ch->top_addr();
a61af66fc99e Initial load duke parents: diff changeset	1000 address end_adr = (address)ch->end_addr();
a61af66fc99e Initial load duke parents: diff changeset	1001 eden_top_adr = makecon(TypeRawPtr::make(top_adr));
a61af66fc99e Initial load duke parents: diff changeset	1002 eden_end_adr = basic_plus_adr(eden_top_adr, end_adr - top_adr);
a61af66fc99e Initial load duke parents: diff changeset	1003 }
a61af66fc99e Initial load duke parents: diff changeset	1004 }
a61af66fc99e Initial load duke parents: diff changeset	1005
a61af66fc99e Initial load duke parents: diff changeset	1006
a61af66fc99e Initial load duke parents: diff changeset	1007 Node* PhaseMacroExpand::make_load(Node* ctl, Node* mem, Node* base, int offset, const Type* value_type, BasicType bt) {
a61af66fc99e Initial load duke parents: diff changeset	1008 Node* adr = basic_plus_adr(base, offset);
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1009 const TypePtr* adr_type = adr->bottom_type()->is_ptr();
113 ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 73 diff changeset	1010 Node* value = LoadNode::make(_igvn, ctl, mem, adr, adr_type, value_type, bt);
0 a61af66fc99e Initial load duke parents: diff changeset	1011 transform_later(value);
a61af66fc99e Initial load duke parents: diff changeset	1012 return value;
a61af66fc99e Initial load duke parents: diff changeset	1013 }
a61af66fc99e Initial load duke parents: diff changeset	1014
a61af66fc99e Initial load duke parents: diff changeset	1015
a61af66fc99e Initial load duke parents: diff changeset	1016 Node* PhaseMacroExpand::make_store(Node* ctl, Node* mem, Node* base, int offset, Node* value, BasicType bt) {
a61af66fc99e Initial load duke parents: diff changeset	1017 Node* adr = basic_plus_adr(base, offset);
113 ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 73 diff changeset	1018 mem = StoreNode::make(_igvn, ctl, mem, adr, NULL, value, bt);
0 a61af66fc99e Initial load duke parents: diff changeset	1019 transform_later(mem);
a61af66fc99e Initial load duke parents: diff changeset	1020 return mem;
a61af66fc99e Initial load duke parents: diff changeset	1021 }
a61af66fc99e Initial load duke parents: diff changeset	1022
a61af66fc99e Initial load duke parents: diff changeset	1023 //=============================================================================
a61af66fc99e Initial load duke parents: diff changeset	1024 //
a61af66fc99e Initial load duke parents: diff changeset	1025 // A L L O C A T I O N
a61af66fc99e Initial load duke parents: diff changeset	1026 //
a61af66fc99e Initial load duke parents: diff changeset	1027 // Allocation attempts to be fast in the case of frequent small objects.
a61af66fc99e Initial load duke parents: diff changeset	1028 // It breaks down like this:
a61af66fc99e Initial load duke parents: diff changeset	1029 //
a61af66fc99e Initial load duke parents: diff changeset	1030 // 1) Size in doublewords is computed. This is a constant for objects and
a61af66fc99e Initial load duke parents: diff changeset	1031 // variable for most arrays. Doubleword units are used to avoid size
a61af66fc99e Initial load duke parents: diff changeset	1032 // overflow of huge doubleword arrays. We need doublewords in the end for
a61af66fc99e Initial load duke parents: diff changeset	1033 // rounding.
a61af66fc99e Initial load duke parents: diff changeset	1034 //
a61af66fc99e Initial load duke parents: diff changeset	1035 // 2) Size is checked for being 'too large'. Too-large allocations will go
a61af66fc99e Initial load duke parents: diff changeset	1036 // the slow path into the VM. The slow path can throw any required
a61af66fc99e Initial load duke parents: diff changeset	1037 // exceptions, and does all the special checks for very large arrays. The
a61af66fc99e Initial load duke parents: diff changeset	1038 // size test can constant-fold away for objects. For objects with
a61af66fc99e Initial load duke parents: diff changeset	1039 // finalizers it constant-folds the otherway: you always go slow with
a61af66fc99e Initial load duke parents: diff changeset	1040 // finalizers.
a61af66fc99e Initial load duke parents: diff changeset	1041 //
a61af66fc99e Initial load duke parents: diff changeset	1042 // 3) If NOT using TLABs, this is the contended loop-back point.
a61af66fc99e Initial load duke parents: diff changeset	1043 // Load-Locked the heap top. If using TLABs normal-load the heap top.
a61af66fc99e Initial load duke parents: diff changeset	1044 //
a61af66fc99e Initial load duke parents: diff changeset	1045 // 4) Check that heap top + size*8 < max. If we fail go the slow ` route.
a61af66fc99e Initial load duke parents: diff changeset	1046 // NOTE: "top+size*8" cannot wrap the 4Gig line! Here's why: for largish
a61af66fc99e Initial load duke parents: diff changeset	1047 // "size*8" we always enter the VM, where "largish" is a constant picked small
a61af66fc99e Initial load duke parents: diff changeset	1048 // enough that there's always space between the eden max and 4Gig (old space is
a61af66fc99e Initial load duke parents: diff changeset	1049 // there so it's quite large) and large enough that the cost of entering the VM
a61af66fc99e Initial load duke parents: diff changeset	1050 // is dwarfed by the cost to initialize the space.
a61af66fc99e Initial load duke parents: diff changeset	1051 //
a61af66fc99e Initial load duke parents: diff changeset	1052 // 5) If NOT using TLABs, Store-Conditional the adjusted heap top back
a61af66fc99e Initial load duke parents: diff changeset	1053 // down. If contended, repeat at step 3. If using TLABs normal-store
a61af66fc99e Initial load duke parents: diff changeset	1054 // adjusted heap top back down; there is no contention.
a61af66fc99e Initial load duke parents: diff changeset	1055 //
a61af66fc99e Initial load duke parents: diff changeset	1056 // 6) If !ZeroTLAB then Bulk-clear the object/array. Fill in klass & mark
a61af66fc99e Initial load duke parents: diff changeset	1057 // fields.
a61af66fc99e Initial load duke parents: diff changeset	1058 //
a61af66fc99e Initial load duke parents: diff changeset	1059 // 7) Merge with the slow-path; cast the raw memory pointer to the correct
a61af66fc99e Initial load duke parents: diff changeset	1060 // oop flavor.
a61af66fc99e Initial load duke parents: diff changeset	1061 //
a61af66fc99e Initial load duke parents: diff changeset	1062 //=============================================================================
a61af66fc99e Initial load duke parents: diff changeset	1063 // FastAllocateSizeLimit value is in DOUBLEWORDS.
a61af66fc99e Initial load duke parents: diff changeset	1064 // Allocations bigger than this always go the slow route.
a61af66fc99e Initial load duke parents: diff changeset	1065 // This value must be small enough that allocation attempts that need to
a61af66fc99e Initial load duke parents: diff changeset	1066 // trigger exceptions go the slow route. Also, it must be small enough so
a61af66fc99e Initial load duke parents: diff changeset	1067 // that heap_top + size_in_bytes does not wrap around the 4Gig limit.
a61af66fc99e Initial load duke parents: diff changeset	1068 //=============================================================================j//
a61af66fc99e Initial load duke parents: diff changeset	1069 // %%% Here is an old comment from parseHelper.cpp; is it outdated?
a61af66fc99e Initial load duke parents: diff changeset	1070 // The allocator will coalesce int->oop copies away. See comment in
a61af66fc99e Initial load duke parents: diff changeset	1071 // coalesce.cpp about how this works. It depends critically on the exact
a61af66fc99e Initial load duke parents: diff changeset	1072 // code shape produced here, so if you are changing this code shape
a61af66fc99e Initial load duke parents: diff changeset	1073 // make sure the GC info for the heap-top is correct in and around the
a61af66fc99e Initial load duke parents: diff changeset	1074 // slow-path call.
a61af66fc99e Initial load duke parents: diff changeset	1075 //
a61af66fc99e Initial load duke parents: diff changeset	1076
a61af66fc99e Initial load duke parents: diff changeset	1077 void PhaseMacroExpand::expand_allocate_common(
a61af66fc99e Initial load duke parents: diff changeset	1078 AllocateNode* alloc, // allocation node to be expanded
a61af66fc99e Initial load duke parents: diff changeset	1079 Node* length, // array length for an array allocation
a61af66fc99e Initial load duke parents: diff changeset	1080 const TypeFunc* slow_call_type, // Type of slow call
a61af66fc99e Initial load duke parents: diff changeset	1081 address slow_call_address // Address of slow call
a61af66fc99e Initial load duke parents: diff changeset	1082 )
a61af66fc99e Initial load duke parents: diff changeset	1083 {
a61af66fc99e Initial load duke parents: diff changeset	1084
a61af66fc99e Initial load duke parents: diff changeset	1085 Node* ctrl = alloc->in(TypeFunc::Control);
a61af66fc99e Initial load duke parents: diff changeset	1086 Node* mem = alloc->in(TypeFunc::Memory);
a61af66fc99e Initial load duke parents: diff changeset	1087 Node* i_o = alloc->in(TypeFunc::I_O);
a61af66fc99e Initial load duke parents: diff changeset	1088 Node* size_in_bytes = alloc->in(AllocateNode::AllocSize);
a61af66fc99e Initial load duke parents: diff changeset	1089 Node* klass_node = alloc->in(AllocateNode::KlassNode);
a61af66fc99e Initial load duke parents: diff changeset	1090 Node* initial_slow_test = alloc->in(AllocateNode::InitialTest);
a61af66fc99e Initial load duke parents: diff changeset	1091
a61af66fc99e Initial load duke parents: diff changeset	1092 assert(ctrl != NULL, "must have control");
a61af66fc99e Initial load duke parents: diff changeset	1093 // We need a Region and corresponding Phi's to merge the slow-path and fast-path results.
a61af66fc99e Initial load duke parents: diff changeset	1094 // they will not be used if "always_slow" is set
a61af66fc99e Initial load duke parents: diff changeset	1095 enum { slow_result_path = 1, fast_result_path = 2 };
a61af66fc99e Initial load duke parents: diff changeset	1096 Node *result_region;
a61af66fc99e Initial load duke parents: diff changeset	1097 Node *result_phi_rawmem;
a61af66fc99e Initial load duke parents: diff changeset	1098 Node *result_phi_rawoop;
a61af66fc99e Initial load duke parents: diff changeset	1099 Node *result_phi_i_o;
a61af66fc99e Initial load duke parents: diff changeset	1100
a61af66fc99e Initial load duke parents: diff changeset	1101 // The initial slow comparison is a size check, the comparison
a61af66fc99e Initial load duke parents: diff changeset	1102 // we want to do is a BoolTest::gt
a61af66fc99e Initial load duke parents: diff changeset	1103 bool always_slow = false;
a61af66fc99e Initial load duke parents: diff changeset	1104 int tv = _igvn.find_int_con(initial_slow_test, -1);
a61af66fc99e Initial load duke parents: diff changeset	1105 if (tv >= 0) {
a61af66fc99e Initial load duke parents: diff changeset	1106 always_slow = (tv == 1);
a61af66fc99e Initial load duke parents: diff changeset	1107 initial_slow_test = NULL;
a61af66fc99e Initial load duke parents: diff changeset	1108 } else {
a61af66fc99e Initial load duke parents: diff changeset	1109 initial_slow_test = BoolNode::make_predicate(initial_slow_test, &_igvn);
a61af66fc99e Initial load duke parents: diff changeset	1110 }
a61af66fc99e Initial load duke parents: diff changeset	1111
780 c96bf21b756f 6788527: Server vm intermittently fails with assertion "live value must not be garbage" with fastdebug bits kvn parents: 708 diff changeset	1112 if (C->env()->dtrace_alloc_probes() \|\|
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1113 !UseTLAB && (!Universe::heap()->supports_inline_contig_alloc() \|\|
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1114 (UseConcMarkSweepGC && CMSIncrementalMode))) {
0 a61af66fc99e Initial load duke parents: diff changeset	1115 // Force slow-path allocation
a61af66fc99e Initial load duke parents: diff changeset	1116 always_slow = true;
a61af66fc99e Initial load duke parents: diff changeset	1117 initial_slow_test = NULL;
a61af66fc99e Initial load duke parents: diff changeset	1118 }
a61af66fc99e Initial load duke parents: diff changeset	1119
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1120
0 a61af66fc99e Initial load duke parents: diff changeset	1121 enum { too_big_or_final_path = 1, need_gc_path = 2 };
a61af66fc99e Initial load duke parents: diff changeset	1122 Node *slow_region = NULL;
a61af66fc99e Initial load duke parents: diff changeset	1123 Node *toobig_false = ctrl;
a61af66fc99e Initial load duke parents: diff changeset	1124
a61af66fc99e Initial load duke parents: diff changeset	1125 assert (initial_slow_test == NULL \|\| !always_slow, "arguments must be consistent");
a61af66fc99e Initial load duke parents: diff changeset	1126 // generate the initial test if necessary
a61af66fc99e Initial load duke parents: diff changeset	1127 if (initial_slow_test != NULL ) {
a61af66fc99e Initial load duke parents: diff changeset	1128 slow_region = new (C, 3) RegionNode(3);
a61af66fc99e Initial load duke parents: diff changeset	1129
a61af66fc99e Initial load duke parents: diff changeset	1130 // Now make the initial failure test. Usually a too-big test but
a61af66fc99e Initial load duke parents: diff changeset	1131 // might be a TRUE for finalizers or a fancy class check for
a61af66fc99e Initial load duke parents: diff changeset	1132 // newInstance0.
a61af66fc99e Initial load duke parents: diff changeset	1133 IfNode *toobig_iff = new (C, 2) IfNode(ctrl, initial_slow_test, PROB_MIN, COUNT_UNKNOWN);
a61af66fc99e Initial load duke parents: diff changeset	1134 transform_later(toobig_iff);
a61af66fc99e Initial load duke parents: diff changeset	1135 // Plug the failing-too-big test into the slow-path region
a61af66fc99e Initial load duke parents: diff changeset	1136 Node *toobig_true = new (C, 1) IfTrueNode( toobig_iff );
a61af66fc99e Initial load duke parents: diff changeset	1137 transform_later(toobig_true);
a61af66fc99e Initial load duke parents: diff changeset	1138 slow_region ->init_req( too_big_or_final_path, toobig_true );
a61af66fc99e Initial load duke parents: diff changeset	1139 toobig_false = new (C, 1) IfFalseNode( toobig_iff );
a61af66fc99e Initial load duke parents: diff changeset	1140 transform_later(toobig_false);
a61af66fc99e Initial load duke parents: diff changeset	1141 } else { // No initial test, just fall into next case
a61af66fc99e Initial load duke parents: diff changeset	1142 toobig_false = ctrl;
a61af66fc99e Initial load duke parents: diff changeset	1143 debug_only(slow_region = NodeSentinel);
a61af66fc99e Initial load duke parents: diff changeset	1144 }
a61af66fc99e Initial load duke parents: diff changeset	1145
a61af66fc99e Initial load duke parents: diff changeset	1146 Node *slow_mem = mem; // save the current memory state for slow path
a61af66fc99e Initial load duke parents: diff changeset	1147 // generate the fast allocation code unless we know that the initial test will always go slow
a61af66fc99e Initial load duke parents: diff changeset	1148 if (!always_slow) {
565 7fe62bb75bf4 6799693: Server compiler leads to data corruption when expression throws an Exception kvn parents: 490 diff changeset	1149 // Fast path modifies only raw memory.
7fe62bb75bf4 6799693: Server compiler leads to data corruption when expression throws an Exception kvn parents: 490 diff changeset	1150 if (mem->is_MergeMem()) {
7fe62bb75bf4 6799693: Server compiler leads to data corruption when expression throws an Exception kvn parents: 490 diff changeset	1151 mem = mem->as_MergeMem()->memory_at(Compile::AliasIdxRaw);
7fe62bb75bf4 6799693: Server compiler leads to data corruption when expression throws an Exception kvn parents: 490 diff changeset	1152 }
7fe62bb75bf4 6799693: Server compiler leads to data corruption when expression throws an Exception kvn parents: 490 diff changeset	1153
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1154 Node* eden_top_adr;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1155 Node* eden_end_adr;
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1156
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1157 set_eden_pointers(eden_top_adr, eden_end_adr);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1158
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1159 // Load Eden::end. Loop invariant and hoisted.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1160 //
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1161 // Note: We set the control input on "eden_end" and "old_eden_top" when using
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1162 // a TLAB to work around a bug where these values were being moved across
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1163 // a safepoint. These are not oops, so they cannot be include in the oop
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1164 // map, but they can be changed by a GC. The proper way to fix this would
342 37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1165 // be to set the raw memory state when generating a SafepointNode. However
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1166 // this will require extensive changes to the loop optimization in order to
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1167 // prevent a degradation of the optimization.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1168 // See comment in memnode.hpp, around line 227 in class LoadPNode.
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1169 Node *eden_end = make_load(ctrl, mem, eden_end_adr, 0, TypeRawPtr::BOTTOM, T_ADDRESS);
37f87013dfd8 6711316: Open source the Garbage-First garbage collector ysr parents: 124 diff changeset	1170
0 a61af66fc99e Initial load duke parents: diff changeset	1171 // allocate the Region and Phi nodes for the result
a61af66fc99e Initial load duke parents: diff changeset	1172 result_region = new (C, 3) RegionNode(3);
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1173 result_phi_rawmem = new (C, 3) PhiNode(result_region, Type::MEMORY, TypeRawPtr::BOTTOM);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1174 result_phi_rawoop = new (C, 3) PhiNode(result_region, TypeRawPtr::BOTTOM);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1175 result_phi_i_o = new (C, 3) PhiNode(result_region, Type::ABIO); // I/O is used for Prefetch
0 a61af66fc99e Initial load duke parents: diff changeset	1176
a61af66fc99e Initial load duke parents: diff changeset	1177 // We need a Region for the loop-back contended case.
a61af66fc99e Initial load duke parents: diff changeset	1178 enum { fall_in_path = 1, contended_loopback_path = 2 };
a61af66fc99e Initial load duke parents: diff changeset	1179 Node *contended_region;
a61af66fc99e Initial load duke parents: diff changeset	1180 Node *contended_phi_rawmem;
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1181 if (UseTLAB) {
0 a61af66fc99e Initial load duke parents: diff changeset	1182 contended_region = toobig_false;
a61af66fc99e Initial load duke parents: diff changeset	1183 contended_phi_rawmem = mem;
a61af66fc99e Initial load duke parents: diff changeset	1184 } else {
a61af66fc99e Initial load duke parents: diff changeset	1185 contended_region = new (C, 3) RegionNode(3);
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1186 contended_phi_rawmem = new (C, 3) PhiNode(contended_region, Type::MEMORY, TypeRawPtr::BOTTOM);
0 a61af66fc99e Initial load duke parents: diff changeset	1187 // Now handle the passing-too-big test. We fall into the contended
a61af66fc99e Initial load duke parents: diff changeset	1188 // loop-back merge point.
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1189 contended_region ->init_req(fall_in_path, toobig_false);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1190 contended_phi_rawmem->init_req(fall_in_path, mem);
0 a61af66fc99e Initial load duke parents: diff changeset	1191 transform_later(contended_region);
a61af66fc99e Initial load duke parents: diff changeset	1192 transform_later(contended_phi_rawmem);
a61af66fc99e Initial load duke parents: diff changeset	1193 }
a61af66fc99e Initial load duke parents: diff changeset	1194
a61af66fc99e Initial load duke parents: diff changeset	1195 // Load(-locked) the heap top.
a61af66fc99e Initial load duke parents: diff changeset	1196 // See note above concerning the control input when using a TLAB
a61af66fc99e Initial load duke parents: diff changeset	1197 Node *old_eden_top = UseTLAB
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1198 ? new (C, 3) LoadPNode (ctrl, contended_phi_rawmem, eden_top_adr, TypeRawPtr::BOTTOM, TypeRawPtr::BOTTOM)
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1199 : new (C, 3) LoadPLockedNode(contended_region, contended_phi_rawmem, eden_top_adr);
0 a61af66fc99e Initial load duke parents: diff changeset	1200
a61af66fc99e Initial load duke parents: diff changeset	1201 transform_later(old_eden_top);
a61af66fc99e Initial load duke parents: diff changeset	1202 // Add to heap top to get a new heap top
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1203 Node *new_eden_top = new (C, 4) AddPNode(top(), old_eden_top, size_in_bytes);
0 a61af66fc99e Initial load duke parents: diff changeset	1204 transform_later(new_eden_top);
a61af66fc99e Initial load duke parents: diff changeset	1205 // Check for needing a GC; compare against heap end
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1206 Node *needgc_cmp = new (C, 3) CmpPNode(new_eden_top, eden_end);
0 a61af66fc99e Initial load duke parents: diff changeset	1207 transform_later(needgc_cmp);
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1208 Node *needgc_bol = new (C, 2) BoolNode(needgc_cmp, BoolTest::ge);
0 a61af66fc99e Initial load duke parents: diff changeset	1209 transform_later(needgc_bol);
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1210 IfNode *needgc_iff = new (C, 2) IfNode(contended_region, needgc_bol, PROB_UNLIKELY_MAG(4), COUNT_UNKNOWN);
0 a61af66fc99e Initial load duke parents: diff changeset	1211 transform_later(needgc_iff);
a61af66fc99e Initial load duke parents: diff changeset	1212
a61af66fc99e Initial load duke parents: diff changeset	1213 // Plug the failing-heap-space-need-gc test into the slow-path region
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1214 Node *needgc_true = new (C, 1) IfTrueNode(needgc_iff);
0 a61af66fc99e Initial load duke parents: diff changeset	1215 transform_later(needgc_true);
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1216 if (initial_slow_test) {
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1217 slow_region->init_req(need_gc_path, needgc_true);
0 a61af66fc99e Initial load duke parents: diff changeset	1218 // This completes all paths into the slow merge point
a61af66fc99e Initial load duke parents: diff changeset	1219 transform_later(slow_region);
a61af66fc99e Initial load duke parents: diff changeset	1220 } else { // No initial slow path needed!
a61af66fc99e Initial load duke parents: diff changeset	1221 // Just fall from the need-GC path straight into the VM call.
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1222 slow_region = needgc_true;
0 a61af66fc99e Initial load duke parents: diff changeset	1223 }
a61af66fc99e Initial load duke parents: diff changeset	1224 // No need for a GC. Setup for the Store-Conditional
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1225 Node *needgc_false = new (C, 1) IfFalseNode(needgc_iff);
0 a61af66fc99e Initial load duke parents: diff changeset	1226 transform_later(needgc_false);
a61af66fc99e Initial load duke parents: diff changeset	1227
a61af66fc99e Initial load duke parents: diff changeset	1228 // Grab regular I/O before optional prefetch may change it.
a61af66fc99e Initial load duke parents: diff changeset	1229 // Slow-path does no I/O so just set it to the original I/O.
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1230 result_phi_i_o->init_req(slow_result_path, i_o);
0 a61af66fc99e Initial load duke parents: diff changeset	1231
a61af66fc99e Initial load duke parents: diff changeset	1232 i_o = prefetch_allocation(i_o, needgc_false, contended_phi_rawmem,
a61af66fc99e Initial load duke parents: diff changeset	1233 old_eden_top, new_eden_top, length);
a61af66fc99e Initial load duke parents: diff changeset	1234
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1235 // Name successful fast-path variables
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1236 Node* fast_oop = old_eden_top;
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1237 Node* fast_oop_ctrl;
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1238 Node* fast_oop_rawmem;
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1239
0 a61af66fc99e Initial load duke parents: diff changeset	1240 // Store (-conditional) the modified eden top back down.
a61af66fc99e Initial load duke parents: diff changeset	1241 // StorePConditional produces flags for a test PLUS a modified raw
a61af66fc99e Initial load duke parents: diff changeset	1242 // memory state.
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1243 if (UseTLAB) {
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1244 Node* store_eden_top =
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1245 new (C, 4) StorePNode(needgc_false, contended_phi_rawmem, eden_top_adr,
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1246 TypeRawPtr::BOTTOM, new_eden_top);
0 a61af66fc99e Initial load duke parents: diff changeset	1247 transform_later(store_eden_top);
a61af66fc99e Initial load duke parents: diff changeset	1248 fast_oop_ctrl = needgc_false; // No contention, so this is the fast path
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1249 fast_oop_rawmem = store_eden_top;
0 a61af66fc99e Initial load duke parents: diff changeset	1250 } else {
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1251 Node* store_eden_top =
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1252 new (C, 5) StorePConditionalNode(needgc_false, contended_phi_rawmem, eden_top_adr,
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1253 new_eden_top, fast_oop/old_eden_top/);
0 a61af66fc99e Initial load duke parents: diff changeset	1254 transform_later(store_eden_top);
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1255 Node *contention_check = new (C, 2) BoolNode(store_eden_top, BoolTest::ne);
0 a61af66fc99e Initial load duke parents: diff changeset	1256 transform_later(contention_check);
a61af66fc99e Initial load duke parents: diff changeset	1257 store_eden_top = new (C, 1) SCMemProjNode(store_eden_top);
a61af66fc99e Initial load duke parents: diff changeset	1258 transform_later(store_eden_top);
a61af66fc99e Initial load duke parents: diff changeset	1259
a61af66fc99e Initial load duke parents: diff changeset	1260 // If not using TLABs, check to see if there was contention.
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1261 IfNode *contention_iff = new (C, 2) IfNode (needgc_false, contention_check, PROB_MIN, COUNT_UNKNOWN);
0 a61af66fc99e Initial load duke parents: diff changeset	1262 transform_later(contention_iff);
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1263 Node *contention_true = new (C, 1) IfTrueNode(contention_iff);
0 a61af66fc99e Initial load duke parents: diff changeset	1264 transform_later(contention_true);
a61af66fc99e Initial load duke parents: diff changeset	1265 // If contention, loopback and try again.
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1266 contended_region->init_req(contended_loopback_path, contention_true);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1267 contended_phi_rawmem->init_req(contended_loopback_path, store_eden_top);
0 a61af66fc99e Initial load duke parents: diff changeset	1268
a61af66fc99e Initial load duke parents: diff changeset	1269 // Fast-path succeeded with no contention!
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1270 Node *contention_false = new (C, 1) IfFalseNode(contention_iff);
0 a61af66fc99e Initial load duke parents: diff changeset	1271 transform_later(contention_false);
a61af66fc99e Initial load duke parents: diff changeset	1272 fast_oop_ctrl = contention_false;
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1273
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1274 // Bump total allocated bytes for this thread
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1275 Node* thread = new (C, 1) ThreadLocalNode();
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1276 transform_later(thread);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1277 Node* alloc_bytes_adr = basic_plus_adr(top()/not oop/, thread,
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1278 in_bytes(JavaThread::allocated_bytes_offset()));
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1279 Node* alloc_bytes = make_load(fast_oop_ctrl, store_eden_top, alloc_bytes_adr,
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1280 0, TypeLong::LONG, T_LONG);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1281 #ifdef _LP64
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1282 Node* alloc_size = size_in_bytes;
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1283 #else
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1284 Node* alloc_size = new (C, 2) ConvI2LNode(size_in_bytes);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1285 transform_later(alloc_size);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1286 #endif
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1287 Node* new_alloc_bytes = new (C, 3) AddLNode(alloc_bytes, alloc_size);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1288 transform_later(new_alloc_bytes);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1289 fast_oop_rawmem = make_store(fast_oop_ctrl, store_eden_top, alloc_bytes_adr,
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1290 0, new_alloc_bytes, T_LONG);
0 a61af66fc99e Initial load duke parents: diff changeset	1291 }
a61af66fc99e Initial load duke parents: diff changeset	1292
a61af66fc99e Initial load duke parents: diff changeset	1293 fast_oop_rawmem = initialize_object(alloc,
a61af66fc99e Initial load duke parents: diff changeset	1294 fast_oop_ctrl, fast_oop_rawmem, fast_oop,
a61af66fc99e Initial load duke parents: diff changeset	1295 klass_node, length, size_in_bytes);
a61af66fc99e Initial load duke parents: diff changeset	1296
780 c96bf21b756f 6788527: Server vm intermittently fails with assertion "live value must not be garbage" with fastdebug bits kvn parents: 708 diff changeset	1297 if (C->env()->dtrace_extended_probes()) {
0 a61af66fc99e Initial load duke parents: diff changeset	1298 // Slow-path call
a61af66fc99e Initial load duke parents: diff changeset	1299 int size = TypeFunc::Parms + 2;
a61af66fc99e Initial load duke parents: diff changeset	1300 CallLeafNode *call = new (C, size) CallLeafNode(OptoRuntime::dtrace_object_alloc_Type(),
a61af66fc99e Initial load duke parents: diff changeset	1301 CAST_FROM_FN_PTR(address, SharedRuntime::dtrace_object_alloc_base),
a61af66fc99e Initial load duke parents: diff changeset	1302 "dtrace_object_alloc",
a61af66fc99e Initial load duke parents: diff changeset	1303 TypeRawPtr::BOTTOM);
a61af66fc99e Initial load duke parents: diff changeset	1304
a61af66fc99e Initial load duke parents: diff changeset	1305 // Get base of thread-local storage area
a61af66fc99e Initial load duke parents: diff changeset	1306 Node* thread = new (C, 1) ThreadLocalNode();
a61af66fc99e Initial load duke parents: diff changeset	1307 transform_later(thread);
a61af66fc99e Initial load duke parents: diff changeset	1308
a61af66fc99e Initial load duke parents: diff changeset	1309 call->init_req(TypeFunc::Parms+0, thread);
a61af66fc99e Initial load duke parents: diff changeset	1310 call->init_req(TypeFunc::Parms+1, fast_oop);
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1311 call->init_req(TypeFunc::Control, fast_oop_ctrl);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1312 call->init_req(TypeFunc::I_O , top()); // does no i/o
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1313 call->init_req(TypeFunc::Memory , fast_oop_rawmem);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1314 call->init_req(TypeFunc::ReturnAdr, alloc->in(TypeFunc::ReturnAdr));
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1315 call->init_req(TypeFunc::FramePtr, alloc->in(TypeFunc::FramePtr));
0 a61af66fc99e Initial load duke parents: diff changeset	1316 transform_later(call);
a61af66fc99e Initial load duke parents: diff changeset	1317 fast_oop_ctrl = new (C, 1) ProjNode(call,TypeFunc::Control);
a61af66fc99e Initial load duke parents: diff changeset	1318 transform_later(fast_oop_ctrl);
a61af66fc99e Initial load duke parents: diff changeset	1319 fast_oop_rawmem = new (C, 1) ProjNode(call,TypeFunc::Memory);
a61af66fc99e Initial load duke parents: diff changeset	1320 transform_later(fast_oop_rawmem);
a61af66fc99e Initial load duke parents: diff changeset	1321 }
a61af66fc99e Initial load duke parents: diff changeset	1322
a61af66fc99e Initial load duke parents: diff changeset	1323 // Plug in the successful fast-path into the result merge point
2100 b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1324 result_region ->init_req(fast_result_path, fast_oop_ctrl);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1325 result_phi_rawoop->init_req(fast_result_path, fast_oop);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1326 result_phi_i_o ->init_req(fast_result_path, i_o);
b1a2afa37ec4 7003271: Hotspot should track cumulative Java heap bytes allocated on a per-thread basis phh parents: 1972 diff changeset	1327 result_phi_rawmem->init_req(fast_result_path, fast_oop_rawmem);
0 a61af66fc99e Initial load duke parents: diff changeset	1328 } else {
a61af66fc99e Initial load duke parents: diff changeset	1329 slow_region = ctrl;
a61af66fc99e Initial load duke parents: diff changeset	1330 }
a61af66fc99e Initial load duke parents: diff changeset	1331
a61af66fc99e Initial load duke parents: diff changeset	1332 // Generate slow-path call
a61af66fc99e Initial load duke parents: diff changeset	1333 CallNode *call = new (C, slow_call_type->domain()->cnt())
a61af66fc99e Initial load duke parents: diff changeset	1334 CallStaticJavaNode(slow_call_type, slow_call_address,
a61af66fc99e Initial load duke parents: diff changeset	1335 OptoRuntime::stub_name(slow_call_address),
a61af66fc99e Initial load duke parents: diff changeset	1336 alloc->jvms()->bci(),
a61af66fc99e Initial load duke parents: diff changeset	1337 TypePtr::BOTTOM);
a61af66fc99e Initial load duke parents: diff changeset	1338 call->init_req( TypeFunc::Control, slow_region );
a61af66fc99e Initial load duke parents: diff changeset	1339 call->init_req( TypeFunc::I_O , top() ) ; // does no i/o
a61af66fc99e Initial load duke parents: diff changeset	1340 call->init_req( TypeFunc::Memory , slow_mem ); // may gc ptrs
a61af66fc99e Initial load duke parents: diff changeset	1341 call->init_req( TypeFunc::ReturnAdr, alloc->in(TypeFunc::ReturnAdr) );
a61af66fc99e Initial load duke parents: diff changeset	1342 call->init_req( TypeFunc::FramePtr, alloc->in(TypeFunc::FramePtr) );
a61af66fc99e Initial load duke parents: diff changeset	1343
a61af66fc99e Initial load duke parents: diff changeset	1344 call->init_req(TypeFunc::Parms+0, klass_node);
a61af66fc99e Initial load duke parents: diff changeset	1345 if (length != NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1346 call->init_req(TypeFunc::Parms+1, length);
a61af66fc99e Initial load duke parents: diff changeset	1347 }
a61af66fc99e Initial load duke parents: diff changeset	1348
a61af66fc99e Initial load duke parents: diff changeset	1349 // Copy debug information and adjust JVMState information, then replace
a61af66fc99e Initial load duke parents: diff changeset	1350 // allocate node with the call
a61af66fc99e Initial load duke parents: diff changeset	1351 copy_call_debug_info((CallNode *) alloc, call);
a61af66fc99e Initial load duke parents: diff changeset	1352 if (!always_slow) {
a61af66fc99e Initial load duke parents: diff changeset	1353 call->set_cnt(PROB_UNLIKELY_MAG(4)); // Same effect as RC_UNCOMMON.
a61af66fc99e Initial load duke parents: diff changeset	1354 }
1621 6027dddc26c6 6677629: PhaseIterGVN::subsume_node() should call hash_delete() and add_users_to_worklist() kvn parents: 1609 diff changeset	1355 _igvn.replace_node(alloc, call);
0 a61af66fc99e Initial load duke parents: diff changeset	1356 transform_later(call);
a61af66fc99e Initial load duke parents: diff changeset	1357
a61af66fc99e Initial load duke parents: diff changeset	1358 // Identify the output projections from the allocate node and
a61af66fc99e Initial load duke parents: diff changeset	1359 // adjust any references to them.
a61af66fc99e Initial load duke parents: diff changeset	1360 // The control and io projections look like:
a61af66fc99e Initial load duke parents: diff changeset	1361 //
a61af66fc99e Initial load duke parents: diff changeset	1362 // v---Proj(ctrl) <-----+ v---CatchProj(ctrl)
a61af66fc99e Initial load duke parents: diff changeset	1363 // Allocate Catch
a61af66fc99e Initial load duke parents: diff changeset	1364 // ^---Proj(io) <-------+ ^---CatchProj(io)
a61af66fc99e Initial load duke parents: diff changeset	1365 //
a61af66fc99e Initial load duke parents: diff changeset	1366 // We are interested in the CatchProj nodes.
a61af66fc99e Initial load duke parents: diff changeset	1367 //
a61af66fc99e Initial load duke parents: diff changeset	1368 extract_call_projections(call);
a61af66fc99e Initial load duke parents: diff changeset	1369
a61af66fc99e Initial load duke parents: diff changeset	1370 // An allocate node has separate memory projections for the uses on the control and i_o paths
a61af66fc99e Initial load duke parents: diff changeset	1371 // Replace uses of the control memory projection with result_phi_rawmem (unless we are only generating a slow call)
a61af66fc99e Initial load duke parents: diff changeset	1372 if (!always_slow && _memproj_fallthrough != NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1373 for (DUIterator_Fast imax, i = _memproj_fallthrough->fast_outs(imax); i < imax; i++) {
a61af66fc99e Initial load duke parents: diff changeset	1374 Node *use = _memproj_fallthrough->fast_out(i);
a61af66fc99e Initial load duke parents: diff changeset	1375 _igvn.hash_delete(use);
a61af66fc99e Initial load duke parents: diff changeset	1376 imax -= replace_input(use, _memproj_fallthrough, result_phi_rawmem);
a61af66fc99e Initial load duke parents: diff changeset	1377 _igvn._worklist.push(use);
a61af66fc99e Initial load duke parents: diff changeset	1378 // back up iterator
a61af66fc99e Initial load duke parents: diff changeset	1379 --i;
a61af66fc99e Initial load duke parents: diff changeset	1380 }
a61af66fc99e Initial load duke parents: diff changeset	1381 }
a61af66fc99e Initial load duke parents: diff changeset	1382 // Now change uses of _memproj_catchall to use _memproj_fallthrough and delete _memproj_catchall so
a61af66fc99e Initial load duke parents: diff changeset	1383 // we end up with a call that has only 1 memory projection
a61af66fc99e Initial load duke parents: diff changeset	1384 if (_memproj_catchall != NULL ) {
a61af66fc99e Initial load duke parents: diff changeset	1385 if (_memproj_fallthrough == NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1386 _memproj_fallthrough = new (C, 1) ProjNode(call, TypeFunc::Memory);
a61af66fc99e Initial load duke parents: diff changeset	1387 transform_later(_memproj_fallthrough);
a61af66fc99e Initial load duke parents: diff changeset	1388 }
a61af66fc99e Initial load duke parents: diff changeset	1389 for (DUIterator_Fast imax, i = _memproj_catchall->fast_outs(imax); i < imax; i++) {
a61af66fc99e Initial load duke parents: diff changeset	1390 Node *use = _memproj_catchall->fast_out(i);
a61af66fc99e Initial load duke parents: diff changeset	1391 _igvn.hash_delete(use);
a61af66fc99e Initial load duke parents: diff changeset	1392 imax -= replace_input(use, _memproj_catchall, _memproj_fallthrough);
a61af66fc99e Initial load duke parents: diff changeset	1393 _igvn._worklist.push(use);
a61af66fc99e Initial load duke parents: diff changeset	1394 // back up iterator
a61af66fc99e Initial load duke parents: diff changeset	1395 --i;
a61af66fc99e Initial load duke parents: diff changeset	1396 }
a61af66fc99e Initial load duke parents: diff changeset	1397 }
a61af66fc99e Initial load duke parents: diff changeset	1398
a61af66fc99e Initial load duke parents: diff changeset	1399 // An allocate node has separate i_o projections for the uses on the control and i_o paths
a61af66fc99e Initial load duke parents: diff changeset	1400 // Replace uses of the control i_o projection with result_phi_i_o (unless we are only generating a slow call)
a61af66fc99e Initial load duke parents: diff changeset	1401 if (_ioproj_fallthrough == NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1402 _ioproj_fallthrough = new (C, 1) ProjNode(call, TypeFunc::I_O);
a61af66fc99e Initial load duke parents: diff changeset	1403 transform_later(_ioproj_fallthrough);
a61af66fc99e Initial load duke parents: diff changeset	1404 } else if (!always_slow) {
a61af66fc99e Initial load duke parents: diff changeset	1405 for (DUIterator_Fast imax, i = _ioproj_fallthrough->fast_outs(imax); i < imax; i++) {
a61af66fc99e Initial load duke parents: diff changeset	1406 Node *use = _ioproj_fallthrough->fast_out(i);
a61af66fc99e Initial load duke parents: diff changeset	1407
a61af66fc99e Initial load duke parents: diff changeset	1408 _igvn.hash_delete(use);
a61af66fc99e Initial load duke parents: diff changeset	1409 imax -= replace_input(use, _ioproj_fallthrough, result_phi_i_o);
a61af66fc99e Initial load duke parents: diff changeset	1410 _igvn._worklist.push(use);
a61af66fc99e Initial load duke parents: diff changeset	1411 // back up iterator
a61af66fc99e Initial load duke parents: diff changeset	1412 --i;
a61af66fc99e Initial load duke parents: diff changeset	1413 }
a61af66fc99e Initial load duke parents: diff changeset	1414 }
a61af66fc99e Initial load duke parents: diff changeset	1415 // Now change uses of _ioproj_catchall to use _ioproj_fallthrough and delete _ioproj_catchall so
a61af66fc99e Initial load duke parents: diff changeset	1416 // we end up with a call that has only 1 control projection
a61af66fc99e Initial load duke parents: diff changeset	1417 if (_ioproj_catchall != NULL ) {
a61af66fc99e Initial load duke parents: diff changeset	1418 for (DUIterator_Fast imax, i = _ioproj_catchall->fast_outs(imax); i < imax; i++) {
a61af66fc99e Initial load duke parents: diff changeset	1419 Node *use = _ioproj_catchall->fast_out(i);
a61af66fc99e Initial load duke parents: diff changeset	1420 _igvn.hash_delete(use);
a61af66fc99e Initial load duke parents: diff changeset	1421 imax -= replace_input(use, _ioproj_catchall, _ioproj_fallthrough);
a61af66fc99e Initial load duke parents: diff changeset	1422 _igvn._worklist.push(use);
a61af66fc99e Initial load duke parents: diff changeset	1423 // back up iterator
a61af66fc99e Initial load duke parents: diff changeset	1424 --i;
a61af66fc99e Initial load duke parents: diff changeset	1425 }
a61af66fc99e Initial load duke parents: diff changeset	1426 }
a61af66fc99e Initial load duke parents: diff changeset	1427
a61af66fc99e Initial load duke parents: diff changeset	1428 // if we generated only a slow call, we are done
a61af66fc99e Initial load duke parents: diff changeset	1429 if (always_slow)
a61af66fc99e Initial load duke parents: diff changeset	1430 return;
a61af66fc99e Initial load duke parents: diff changeset	1431
a61af66fc99e Initial load duke parents: diff changeset	1432
a61af66fc99e Initial load duke parents: diff changeset	1433 if (_fallthroughcatchproj != NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1434 ctrl = _fallthroughcatchproj->clone();
a61af66fc99e Initial load duke parents: diff changeset	1435 transform_later(ctrl);
708 f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	1436 _igvn.replace_node(_fallthroughcatchproj, result_region);
0 a61af66fc99e Initial load duke parents: diff changeset	1437 } else {
a61af66fc99e Initial load duke parents: diff changeset	1438 ctrl = top();
a61af66fc99e Initial load duke parents: diff changeset	1439 }
a61af66fc99e Initial load duke parents: diff changeset	1440 Node *slow_result;
a61af66fc99e Initial load duke parents: diff changeset	1441 if (_resproj == NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1442 // no uses of the allocation result
a61af66fc99e Initial load duke parents: diff changeset	1443 slow_result = top();
a61af66fc99e Initial load duke parents: diff changeset	1444 } else {
a61af66fc99e Initial load duke parents: diff changeset	1445 slow_result = _resproj->clone();
a61af66fc99e Initial load duke parents: diff changeset	1446 transform_later(slow_result);
708 f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	1447 _igvn.replace_node(_resproj, result_phi_rawoop);
0 a61af66fc99e Initial load duke parents: diff changeset	1448 }
a61af66fc99e Initial load duke parents: diff changeset	1449
a61af66fc99e Initial load duke parents: diff changeset	1450 // Plug slow-path into result merge point
a61af66fc99e Initial load duke parents: diff changeset	1451 result_region ->init_req( slow_result_path, ctrl );
a61af66fc99e Initial load duke parents: diff changeset	1452 result_phi_rawoop->init_req( slow_result_path, slow_result);
a61af66fc99e Initial load duke parents: diff changeset	1453 result_phi_rawmem->init_req( slow_result_path, _memproj_fallthrough );
a61af66fc99e Initial load duke parents: diff changeset	1454 transform_later(result_region);
a61af66fc99e Initial load duke parents: diff changeset	1455 transform_later(result_phi_rawoop);
a61af66fc99e Initial load duke parents: diff changeset	1456 transform_later(result_phi_rawmem);
a61af66fc99e Initial load duke parents: diff changeset	1457 transform_later(result_phi_i_o);
a61af66fc99e Initial load duke parents: diff changeset	1458 // This completes all paths into the result merge point
a61af66fc99e Initial load duke parents: diff changeset	1459 }
a61af66fc99e Initial load duke parents: diff changeset	1460
a61af66fc99e Initial load duke parents: diff changeset	1461
a61af66fc99e Initial load duke parents: diff changeset	1462 // Helper for PhaseMacroExpand::expand_allocate_common.
a61af66fc99e Initial load duke parents: diff changeset	1463 // Initializes the newly-allocated storage.
a61af66fc99e Initial load duke parents: diff changeset	1464 Node*
a61af66fc99e Initial load duke parents: diff changeset	1465 PhaseMacroExpand::initialize_object(AllocateNode* alloc,
a61af66fc99e Initial load duke parents: diff changeset	1466 Node* control, Node* rawmem, Node* object,
a61af66fc99e Initial load duke parents: diff changeset	1467 Node* klass_node, Node* length,
a61af66fc99e Initial load duke parents: diff changeset	1468 Node* size_in_bytes) {
a61af66fc99e Initial load duke parents: diff changeset	1469 InitializeNode* init = alloc->initialization();
a61af66fc99e Initial load duke parents: diff changeset	1470 // Store the klass & mark bits
a61af66fc99e Initial load duke parents: diff changeset	1471 Node* mark_node = NULL;
a61af66fc99e Initial load duke parents: diff changeset	1472 // For now only enable fast locking for non-array types
a61af66fc99e Initial load duke parents: diff changeset	1473 if (UseBiasedLocking && (length == NULL)) {
1609 4311f23817fd 6959430: Make sure raw loads have control edge kvn parents: 1552 diff changeset	1474 mark_node = make_load(control, rawmem, klass_node, Klass::prototype_header_offset_in_bytes() + sizeof(oopDesc), TypeRawPtr::BOTTOM, T_ADDRESS);
0 a61af66fc99e Initial load duke parents: diff changeset	1475 } else {
a61af66fc99e Initial load duke parents: diff changeset	1476 mark_node = makecon(TypeRawPtr::make((address)markOopDesc::prototype()));
a61af66fc99e Initial load duke parents: diff changeset	1477 }
a61af66fc99e Initial load duke parents: diff changeset	1478 rawmem = make_store(control, rawmem, object, oopDesc::mark_offset_in_bytes(), mark_node, T_ADDRESS);
113 ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 73 diff changeset	1479
0 a61af66fc99e Initial load duke parents: diff changeset	1480 rawmem = make_store(control, rawmem, object, oopDesc::klass_offset_in_bytes(), klass_node, T_OBJECT);
a61af66fc99e Initial load duke parents: diff changeset	1481 int header_size = alloc->minimum_header_size(); // conservatively small
a61af66fc99e Initial load duke parents: diff changeset	1482
a61af66fc99e Initial load duke parents: diff changeset	1483 // Array length
a61af66fc99e Initial load duke parents: diff changeset	1484 if (length != NULL) { // Arrays need length field
a61af66fc99e Initial load duke parents: diff changeset	1485 rawmem = make_store(control, rawmem, object, arrayOopDesc::length_offset_in_bytes(), length, T_INT);
a61af66fc99e Initial load duke parents: diff changeset	1486 // conservatively small header size:
113 ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 73 diff changeset	1487 header_size = arrayOopDesc::base_offset_in_bytes(T_BYTE);
0 a61af66fc99e Initial load duke parents: diff changeset	1488 ciKlass* k = _igvn.type(klass_node)->is_klassptr()->klass();
a61af66fc99e Initial load duke parents: diff changeset	1489 if (k->is_array_klass()) // we know the exact header size in most cases:
a61af66fc99e Initial load duke parents: diff changeset	1490 header_size = Klass::layout_helper_header_size(k->layout_helper());
a61af66fc99e Initial load duke parents: diff changeset	1491 }
a61af66fc99e Initial load duke parents: diff changeset	1492
a61af66fc99e Initial load duke parents: diff changeset	1493 // Clear the object body, if necessary.
a61af66fc99e Initial load duke parents: diff changeset	1494 if (init == NULL) {
a61af66fc99e Initial load duke parents: diff changeset	1495 // The init has somehow disappeared; be cautious and clear everything.
a61af66fc99e Initial load duke parents: diff changeset	1496 //
a61af66fc99e Initial load duke parents: diff changeset	1497 // This can happen if a node is allocated but an uncommon trap occurs
a61af66fc99e Initial load duke parents: diff changeset	1498 // immediately. In this case, the Initialize gets associated with the
a61af66fc99e Initial load duke parents: diff changeset	1499 // trap, and may be placed in a different (outer) loop, if the Allocate
a61af66fc99e Initial load duke parents: diff changeset	1500 // is in a loop. If (this is rare) the inner loop gets unrolled, then
a61af66fc99e Initial load duke parents: diff changeset	1501 // there can be two Allocates to one Initialize. The answer in all these
a61af66fc99e Initial load duke parents: diff changeset	1502 // edge cases is safety first. It is always safe to clear immediately
a61af66fc99e Initial load duke parents: diff changeset	1503 // within an Allocate, and then (maybe or maybe not) clear some more later.
a61af66fc99e Initial load duke parents: diff changeset	1504 if (!ZeroTLAB)
a61af66fc99e Initial load duke parents: diff changeset	1505 rawmem = ClearArrayNode::clear_memory(control, rawmem, object,
a61af66fc99e Initial load duke parents: diff changeset	1506 header_size, size_in_bytes,
a61af66fc99e Initial load duke parents: diff changeset	1507 &_igvn);
a61af66fc99e Initial load duke parents: diff changeset	1508 } else {
a61af66fc99e Initial load duke parents: diff changeset	1509 if (!init->is_complete()) {
a61af66fc99e Initial load duke parents: diff changeset	1510 // Try to win by zeroing only what the init does not store.
a61af66fc99e Initial load duke parents: diff changeset	1511 // We can also try to do some peephole optimizations,
a61af66fc99e Initial load duke parents: diff changeset	1512 // such as combining some adjacent subword stores.
a61af66fc99e Initial load duke parents: diff changeset	1513 rawmem = init->complete_stores(control, rawmem, object,
a61af66fc99e Initial load duke parents: diff changeset	1514 header_size, size_in_bytes, &_igvn);
a61af66fc99e Initial load duke parents: diff changeset	1515 }
a61af66fc99e Initial load duke parents: diff changeset	1516 // We have no more use for this link, since the AllocateNode goes away:
a61af66fc99e Initial load duke parents: diff changeset	1517 init->set_req(InitializeNode::RawAddress, top());
a61af66fc99e Initial load duke parents: diff changeset	1518 // (If we keep the link, it just confuses the register allocator,
a61af66fc99e Initial load duke parents: diff changeset	1519 // who thinks he sees a real use of the address by the membar.)
a61af66fc99e Initial load duke parents: diff changeset	1520 }
a61af66fc99e Initial load duke parents: diff changeset	1521
a61af66fc99e Initial load duke parents: diff changeset	1522 return rawmem;
a61af66fc99e Initial load duke parents: diff changeset	1523 }
a61af66fc99e Initial load duke parents: diff changeset	1524
a61af66fc99e Initial load duke parents: diff changeset	1525 // Generate prefetch instructions for next allocations.
a61af66fc99e Initial load duke parents: diff changeset	1526 Node* PhaseMacroExpand::prefetch_allocation(Node* i_o, Node*& needgc_false,
a61af66fc99e Initial load duke parents: diff changeset	1527 Node*& contended_phi_rawmem,
a61af66fc99e Initial load duke parents: diff changeset	1528 Node* old_eden_top, Node* new_eden_top,
a61af66fc99e Initial load duke parents: diff changeset	1529 Node* length) {
1367 9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1530 enum { fall_in_path = 1, pf_path = 2 };
0 a61af66fc99e Initial load duke parents: diff changeset	1531 if( UseTLAB && AllocatePrefetchStyle == 2 ) {
a61af66fc99e Initial load duke parents: diff changeset	1532 // Generate prefetch allocation with watermark check.
a61af66fc99e Initial load duke parents: diff changeset	1533 // As an allocation hits the watermark, we will prefetch starting
a61af66fc99e Initial load duke parents: diff changeset	1534 // at a "distance" away from watermark.
a61af66fc99e Initial load duke parents: diff changeset	1535
a61af66fc99e Initial load duke parents: diff changeset	1536 Node *pf_region = new (C, 3) RegionNode(3);
a61af66fc99e Initial load duke parents: diff changeset	1537 Node *pf_phi_rawmem = new (C, 3) PhiNode( pf_region, Type::MEMORY,
a61af66fc99e Initial load duke parents: diff changeset	1538 TypeRawPtr::BOTTOM );
a61af66fc99e Initial load duke parents: diff changeset	1539 // I/O is used for Prefetch
a61af66fc99e Initial load duke parents: diff changeset	1540 Node *pf_phi_abio = new (C, 3) PhiNode( pf_region, Type::ABIO );
a61af66fc99e Initial load duke parents: diff changeset	1541
a61af66fc99e Initial load duke parents: diff changeset	1542 Node *thread = new (C, 1) ThreadLocalNode();
a61af66fc99e Initial load duke parents: diff changeset	1543 transform_later(thread);
a61af66fc99e Initial load duke parents: diff changeset	1544
a61af66fc99e Initial load duke parents: diff changeset	1545 Node eden_pf_adr = new (C, 4) AddPNode( top()/not oop*/, thread,
a61af66fc99e Initial load duke parents: diff changeset	1546 _igvn.MakeConX(in_bytes(JavaThread::tlab_pf_top_offset())) );
a61af66fc99e Initial load duke parents: diff changeset	1547 transform_later(eden_pf_adr);
a61af66fc99e Initial load duke parents: diff changeset	1548
a61af66fc99e Initial load duke parents: diff changeset	1549 Node *old_pf_wm = new (C, 3) LoadPNode( needgc_false,
a61af66fc99e Initial load duke parents: diff changeset	1550 contended_phi_rawmem, eden_pf_adr,
a61af66fc99e Initial load duke parents: diff changeset	1551 TypeRawPtr::BOTTOM, TypeRawPtr::BOTTOM );
a61af66fc99e Initial load duke parents: diff changeset	1552 transform_later(old_pf_wm);
a61af66fc99e Initial load duke parents: diff changeset	1553
a61af66fc99e Initial load duke parents: diff changeset	1554 // check against new_eden_top
a61af66fc99e Initial load duke parents: diff changeset	1555 Node *need_pf_cmp = new (C, 3) CmpPNode( new_eden_top, old_pf_wm );
a61af66fc99e Initial load duke parents: diff changeset	1556 transform_later(need_pf_cmp);
a61af66fc99e Initial load duke parents: diff changeset	1557 Node *need_pf_bol = new (C, 2) BoolNode( need_pf_cmp, BoolTest::ge );
a61af66fc99e Initial load duke parents: diff changeset	1558 transform_later(need_pf_bol);
a61af66fc99e Initial load duke parents: diff changeset	1559 IfNode *need_pf_iff = new (C, 2) IfNode( needgc_false, need_pf_bol,
a61af66fc99e Initial load duke parents: diff changeset	1560 PROB_UNLIKELY_MAG(4), COUNT_UNKNOWN );
a61af66fc99e Initial load duke parents: diff changeset	1561 transform_later(need_pf_iff);
a61af66fc99e Initial load duke parents: diff changeset	1562
a61af66fc99e Initial load duke parents: diff changeset	1563 // true node, add prefetchdistance
a61af66fc99e Initial load duke parents: diff changeset	1564 Node *need_pf_true = new (C, 1) IfTrueNode( need_pf_iff );
a61af66fc99e Initial load duke parents: diff changeset	1565 transform_later(need_pf_true);
a61af66fc99e Initial load duke parents: diff changeset	1566
a61af66fc99e Initial load duke parents: diff changeset	1567 Node *need_pf_false = new (C, 1) IfFalseNode( need_pf_iff );
a61af66fc99e Initial load duke parents: diff changeset	1568 transform_later(need_pf_false);
a61af66fc99e Initial load duke parents: diff changeset	1569
a61af66fc99e Initial load duke parents: diff changeset	1570 Node *new_pf_wmt = new (C, 4) AddPNode( top(), old_pf_wm,
a61af66fc99e Initial load duke parents: diff changeset	1571 _igvn.MakeConX(AllocatePrefetchDistance) );
a61af66fc99e Initial load duke parents: diff changeset	1572 transform_later(new_pf_wmt );
a61af66fc99e Initial load duke parents: diff changeset	1573 new_pf_wmt->set_req(0, need_pf_true);
a61af66fc99e Initial load duke parents: diff changeset	1574
a61af66fc99e Initial load duke parents: diff changeset	1575 Node *store_new_wmt = new (C, 4) StorePNode( need_pf_true,
a61af66fc99e Initial load duke parents: diff changeset	1576 contended_phi_rawmem, eden_pf_adr,
a61af66fc99e Initial load duke parents: diff changeset	1577 TypeRawPtr::BOTTOM, new_pf_wmt );
a61af66fc99e Initial load duke parents: diff changeset	1578 transform_later(store_new_wmt);
a61af66fc99e Initial load duke parents: diff changeset	1579
a61af66fc99e Initial load duke parents: diff changeset	1580 // adding prefetches
a61af66fc99e Initial load duke parents: diff changeset	1581 pf_phi_abio->init_req( fall_in_path, i_o );
a61af66fc99e Initial load duke parents: diff changeset	1582
a61af66fc99e Initial load duke parents: diff changeset	1583 Node *prefetch_adr;
a61af66fc99e Initial load duke parents: diff changeset	1584 Node *prefetch;
a61af66fc99e Initial load duke parents: diff changeset	1585 uint lines = AllocatePrefetchDistance / AllocatePrefetchStepSize;
a61af66fc99e Initial load duke parents: diff changeset	1586 uint step_size = AllocatePrefetchStepSize;
a61af66fc99e Initial load duke parents: diff changeset	1587 uint distance = 0;
a61af66fc99e Initial load duke parents: diff changeset	1588
a61af66fc99e Initial load duke parents: diff changeset	1589 for ( uint i = 0; i < lines; i++ ) {
a61af66fc99e Initial load duke parents: diff changeset	1590 prefetch_adr = new (C, 4) AddPNode( old_pf_wm, new_pf_wmt,
a61af66fc99e Initial load duke parents: diff changeset	1591 _igvn.MakeConX(distance) );
a61af66fc99e Initial load duke parents: diff changeset	1592 transform_later(prefetch_adr);
3854 1af104d6cf99 7079329: Adjust allocation prefetching for T4 kvn parents: 3849 diff changeset	1593 prefetch = new (C, 3) PrefetchAllocationNode( i_o, prefetch_adr );
0 a61af66fc99e Initial load duke parents: diff changeset	1594 transform_later(prefetch);
a61af66fc99e Initial load duke parents: diff changeset	1595 distance += step_size;
a61af66fc99e Initial load duke parents: diff changeset	1596 i_o = prefetch;
a61af66fc99e Initial load duke parents: diff changeset	1597 }
a61af66fc99e Initial load duke parents: diff changeset	1598 pf_phi_abio->set_req( pf_path, i_o );
a61af66fc99e Initial load duke parents: diff changeset	1599
a61af66fc99e Initial load duke parents: diff changeset	1600 pf_region->init_req( fall_in_path, need_pf_false );
a61af66fc99e Initial load duke parents: diff changeset	1601 pf_region->init_req( pf_path, need_pf_true );
a61af66fc99e Initial load duke parents: diff changeset	1602
a61af66fc99e Initial load duke parents: diff changeset	1603 pf_phi_rawmem->init_req( fall_in_path, contended_phi_rawmem );
a61af66fc99e Initial load duke parents: diff changeset	1604 pf_phi_rawmem->init_req( pf_path, store_new_wmt );
a61af66fc99e Initial load duke parents: diff changeset	1605
a61af66fc99e Initial load duke parents: diff changeset	1606 transform_later(pf_region);
a61af66fc99e Initial load duke parents: diff changeset	1607 transform_later(pf_phi_rawmem);
a61af66fc99e Initial load duke parents: diff changeset	1608 transform_later(pf_phi_abio);
a61af66fc99e Initial load duke parents: diff changeset	1609
a61af66fc99e Initial load duke parents: diff changeset	1610 needgc_false = pf_region;
a61af66fc99e Initial load duke parents: diff changeset	1611 contended_phi_rawmem = pf_phi_rawmem;
a61af66fc99e Initial load duke parents: diff changeset	1612 i_o = pf_phi_abio;
1367 9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1613 } else if( UseTLAB && AllocatePrefetchStyle == 3 ) {
3854 1af104d6cf99 7079329: Adjust allocation prefetching for T4 kvn parents: 3849 diff changeset	1614 // Insert a prefetch for each allocation.
1af104d6cf99 7079329: Adjust allocation prefetching for T4 kvn parents: 3849 diff changeset	1615 // This code is used for Sparc with BIS.
1367 9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1616 Node *pf_region = new (C, 3) RegionNode(3);
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1617 Node *pf_phi_rawmem = new (C, 3) PhiNode( pf_region, Type::MEMORY,
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1618 TypeRawPtr::BOTTOM );
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1619
3854 1af104d6cf99 7079329: Adjust allocation prefetching for T4 kvn parents: 3849 diff changeset	1620 // Generate several prefetch instructions.
1af104d6cf99 7079329: Adjust allocation prefetching for T4 kvn parents: 3849 diff changeset	1621 uint lines = (length != NULL) ? AllocatePrefetchLines : AllocateInstancePrefetchLines;
1367 9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1622 uint step_size = AllocatePrefetchStepSize;
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1623 uint distance = AllocatePrefetchDistance;
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1624
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1625 // Next cache address.
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1626 Node *cache_adr = new (C, 4) AddPNode(old_eden_top, old_eden_top,
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1627 _igvn.MakeConX(distance));
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1628 transform_later(cache_adr);
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1629 cache_adr = new (C, 2) CastP2XNode(needgc_false, cache_adr);
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1630 transform_later(cache_adr);
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1631 Node* mask = _igvn.MakeConX(~(intptr_t)(step_size-1));
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1632 cache_adr = new (C, 3) AndXNode(cache_adr, mask);
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1633 transform_later(cache_adr);
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1634 cache_adr = new (C, 2) CastX2PNode(cache_adr);
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1635 transform_later(cache_adr);
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1636
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1637 // Prefetch
3854 1af104d6cf99 7079329: Adjust allocation prefetching for T4 kvn parents: 3849 diff changeset	1638 Node *prefetch = new (C, 3) PrefetchAllocationNode( contended_phi_rawmem, cache_adr );
1367 9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1639 prefetch->set_req(0, needgc_false);
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1640 transform_later(prefetch);
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1641 contended_phi_rawmem = prefetch;
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1642 Node *prefetch_adr;
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1643 distance = step_size;
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1644 for ( uint i = 1; i < lines; i++ ) {
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1645 prefetch_adr = new (C, 4) AddPNode( cache_adr, cache_adr,
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1646 _igvn.MakeConX(distance) );
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1647 transform_later(prefetch_adr);
3854 1af104d6cf99 7079329: Adjust allocation prefetching for T4 kvn parents: 3849 diff changeset	1648 prefetch = new (C, 3) PrefetchAllocationNode( contended_phi_rawmem, prefetch_adr );
1367 9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1649 transform_later(prefetch);
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1650 distance += step_size;
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1651 contended_phi_rawmem = prefetch;
9e321dcfa5b7 6940726: Use BIS instruction for allocation prefetch on Sparc kvn parents: 1100 diff changeset	1652 }
0 a61af66fc99e Initial load duke parents: diff changeset	1653 } else if( AllocatePrefetchStyle > 0 ) {
a61af66fc99e Initial load duke parents: diff changeset	1654 // Insert a prefetch for each allocation only on the fast-path
a61af66fc99e Initial load duke parents: diff changeset	1655 Node *prefetch_adr;
a61af66fc99e Initial load duke parents: diff changeset	1656 Node *prefetch;
3854 1af104d6cf99 7079329: Adjust allocation prefetching for T4 kvn parents: 3849 diff changeset	1657 // Generate several prefetch instructions.
1af104d6cf99 7079329: Adjust allocation prefetching for T4 kvn parents: 3849 diff changeset	1658 uint lines = (length != NULL) ? AllocatePrefetchLines : AllocateInstancePrefetchLines;
0 a61af66fc99e Initial load duke parents: diff changeset	1659 uint step_size = AllocatePrefetchStepSize;
a61af66fc99e Initial load duke parents: diff changeset	1660 uint distance = AllocatePrefetchDistance;
a61af66fc99e Initial load duke parents: diff changeset	1661 for ( uint i = 0; i < lines; i++ ) {
a61af66fc99e Initial load duke parents: diff changeset	1662 prefetch_adr = new (C, 4) AddPNode( old_eden_top, new_eden_top,
a61af66fc99e Initial load duke parents: diff changeset	1663 _igvn.MakeConX(distance) );
a61af66fc99e Initial load duke parents: diff changeset	1664 transform_later(prefetch_adr);
3854 1af104d6cf99 7079329: Adjust allocation prefetching for T4 kvn parents: 3849 diff changeset	1665 prefetch = new (C, 3) PrefetchAllocationNode( i_o, prefetch_adr );
0 a61af66fc99e Initial load duke parents: diff changeset	1666 // Do not let it float too high, since if eden_top == eden_end,
a61af66fc99e Initial load duke parents: diff changeset	1667 // both might be null.
a61af66fc99e Initial load duke parents: diff changeset	1668 if( i == 0 ) { // Set control for first prefetch, next follows it
a61af66fc99e Initial load duke parents: diff changeset	1669 prefetch->init_req(0, needgc_false);
a61af66fc99e Initial load duke parents: diff changeset	1670 }
a61af66fc99e Initial load duke parents: diff changeset	1671 transform_later(prefetch);
a61af66fc99e Initial load duke parents: diff changeset	1672 distance += step_size;
a61af66fc99e Initial load duke parents: diff changeset	1673 i_o = prefetch;
a61af66fc99e Initial load duke parents: diff changeset	1674 }
a61af66fc99e Initial load duke parents: diff changeset	1675 }
a61af66fc99e Initial load duke parents: diff changeset	1676 return i_o;
a61af66fc99e Initial load duke parents: diff changeset	1677 }
a61af66fc99e Initial load duke parents: diff changeset	1678
a61af66fc99e Initial load duke parents: diff changeset	1679
a61af66fc99e Initial load duke parents: diff changeset	1680 void PhaseMacroExpand::expand_allocate(AllocateNode *alloc) {
a61af66fc99e Initial load duke parents: diff changeset	1681 expand_allocate_common(alloc, NULL,
a61af66fc99e Initial load duke parents: diff changeset	1682 OptoRuntime::new_instance_Type(),
a61af66fc99e Initial load duke parents: diff changeset	1683 OptoRuntime::new_instance_Java());
a61af66fc99e Initial load duke parents: diff changeset	1684 }
a61af66fc99e Initial load duke parents: diff changeset	1685
a61af66fc99e Initial load duke parents: diff changeset	1686 void PhaseMacroExpand::expand_allocate_array(AllocateArrayNode *alloc) {
a61af66fc99e Initial load duke parents: diff changeset	1687 Node* length = alloc->in(AllocateNode::ALength);
3961 a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1688 InitializeNode* init = alloc->initialization();
a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1689 Node* klass_node = alloc->in(AllocateNode::KlassNode);
a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1690 ciKlass* k = _igvn.type(klass_node)->is_klassptr()->klass();
a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1691 address slow_call_address; // Address of slow call
a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1692 if (init != NULL && init->is_complete_with_arraycopy() &&
a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1693 k->is_type_array_klass()) {
a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1694 // Don't zero type array during slow allocation in VM since
a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1695 // it will be initialized later by arraycopy in compiled code.
a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1696 slow_call_address = OptoRuntime::new_array_nozero_Java();
a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1697 } else {
a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1698 slow_call_address = OptoRuntime::new_array_Java();
a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1699 }
0 a61af66fc99e Initial load duke parents: diff changeset	1700 expand_allocate_common(alloc, length,
a61af66fc99e Initial load duke parents: diff changeset	1701 OptoRuntime::new_array_Type(),
3961 a92cdbac8b9e 7081933: Use zeroing elimination optimization for large array kvn parents: 3854 diff changeset	1702 slow_call_address);
0 a61af66fc99e Initial load duke parents: diff changeset	1703 }
a61af66fc99e Initial load duke parents: diff changeset	1704
3754 642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1705 //-----------------------mark_eliminated_locking_nodes-----------------------
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1706 // During EA obj may point to several objects but after few ideal graph
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1707 // transformations (CCP) it may point to only one non escaping object
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1708 // (but still using phi), corresponding locks and unlocks will be marked
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1709 // for elimination. Later obj could be replaced with a new node (new phi)
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1710 // and which does not have escape information. And later after some graph
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1711 // reshape other locks and unlocks (which were not marked for elimination
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1712 // before) are connected to this new obj (phi) but they still will not be
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1713 // marked for elimination since new obj has no escape information.
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1714 // Mark all associated (same box and obj) lock and unlock nodes for
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1715 // elimination if some of them marked already.
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1716 void PhaseMacroExpand::mark_eliminated_locking_nodes(AbstractLockNode *alock) {
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1717 if (!alock->is_eliminated()) {
3754 642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1718 return;
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1719 }
3754 642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1720 if (!alock->is_coarsened()) { // Eliminated by EA
460 424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1721 // Create new "eliminated" BoxLock node and use it
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1722 // in monitor debug info for the same object.
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1723 BoxLockNode* oldbox = alock->box_node()->as_BoxLock();
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1724 Node* obj = alock->obj_node();
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1725 if (!oldbox->is_eliminated()) {
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1726 BoxLockNode* newbox = oldbox->clone()->as_BoxLock();
3754 642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1727 // Note: BoxLock node is marked eliminated only here
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1728 // and it is used to indicate that all associated lock
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1729 // and unlock nodes are marked for elimination.
460 424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1730 newbox->set_eliminated();
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1731 transform_later(newbox);
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1732 // Replace old box node with new box for all users
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1733 // of the same object.
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1734 for (uint i = 0; i < oldbox->outcnt();) {
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1735
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1736 bool next_edge = true;
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1737 Node* u = oldbox->raw_out(i);
3754 642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1738 if (u->is_AbstractLock() &&
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1739 u->as_AbstractLock()->obj_node() == obj &&
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1740 u->as_AbstractLock()->box_node() == oldbox) {
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1741 // Mark all associated locks and unlocks.
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1742 u->as_AbstractLock()->set_eliminated();
460 424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1743 _igvn.hash_delete(u);
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1744 u->set_req(TypeFunc::Parms + 1, newbox);
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1745 next_edge = false;
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1746 }
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1747 // Replace old box in monitor debug info.
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1748 if (u->is_SafePoint() && u->as_SafePoint()->jvms()) {
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1749 SafePointNode* sfn = u->as_SafePoint();
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1750 JVMState* youngest_jvms = sfn->jvms();
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1751 int max_depth = youngest_jvms->depth();
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1752 for (int depth = 1; depth <= max_depth; depth++) {
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1753 JVMState* jvms = youngest_jvms->of_depth(depth);
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1754 int num_mon = jvms->nof_monitors();
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1755 // Loop over monitors
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1756 for (int idx = 0; idx < num_mon; idx++) {
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1757 Node* obj_node = sfn->monitor_obj(jvms, idx);
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1758 Node* box_node = sfn->monitor_box(jvms, idx);
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1759 if (box_node == oldbox && obj_node == obj) {
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1760 int j = jvms->monitor_box_offset(idx);
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1761 _igvn.hash_delete(u);
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1762 u->set_req(j, newbox);
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1763 next_edge = false;
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1764 }
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1765 } // for (int idx = 0;
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1766 } // for (int depth = 1;
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1767 } // if (u->is_SafePoint()
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1768 if (next_edge) i++;
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1769 } // for (uint i = 0; i < oldbox->outcnt();)
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1770 } // if (!oldbox->is_eliminated())
3754 642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1771 } // if (!alock->is_coarsened())
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1772 }
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1773
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1774 // we have determined that this lock/unlock can be eliminated, we simply
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1775 // eliminate the node without expanding it.
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1776 //
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1777 // Note: The membar's associated with the lock/unlock are currently not
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1778 // eliminated. This should be investigated as a future enhancement.
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1779 //
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1780 bool PhaseMacroExpand::eliminate_locking_node(AbstractLockNode *alock) {
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1781
3754 642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1782 if (!alock->is_eliminated()) {
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1783 return false;
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1784 }
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1785 #ifdef ASSERT
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1786 if (alock->is_Lock() && !alock->is_coarsened()) {
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1787 // Check that new "eliminated" BoxLock node is created.
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1788 BoxLockNode* oldbox = alock->box_node()->as_BoxLock();
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1789 assert(oldbox->is_eliminated(), "should be done already");
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1790 }
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	1791 #endif
1080 7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1792 CompileLog* log = C->log();
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1793 if (log != NULL) {
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1794 log->head("eliminate_lock lock='%d'",
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1795 alock->is_Lock());
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1796 JVMState* p = alock->jvms();
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1797 while (p != NULL) {
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1798 log->elem("jvms bci='%d' method='%d'", p->bci(), log->identify(p->method()));
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1799 p = p->caller();
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1800 }
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1801 log->tail("eliminate_lock");
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1802 }
7c57aead6d3e 6892658: C2 should optimize some stringbuilder patterns never parents: 851 diff changeset	1803
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1804 #ifndef PRODUCT
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1805 if (PrintEliminateLocks) {
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1806 if (alock->is_Lock()) {
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1807 tty->print_cr("++++ Eliminating: %d Lock", alock->_idx);
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1808 } else {
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1809 tty->print_cr("++++ Eliminating: %d Unlock", alock->_idx);
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1810 }
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1811 }
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1812 #endif
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1813
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1814 Node* mem = alock->in(TypeFunc::Memory);
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1815 Node* ctrl = alock->in(TypeFunc::Control);
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1816
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1817 extract_call_projections(alock);
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1818 // There are 2 projections from the lock. The lock node will
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1819 // be deleted when its last use is subsumed below.
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1820 assert(alock->outcnt() == 2 &&
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1821 _fallthroughproj != NULL &&
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1822 _memproj_fallthrough != NULL,
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1823 "Unexpected projections from Lock/Unlock");
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1824
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1825 Node* fallthroughproj = _fallthroughproj;
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1826 Node* memproj_fallthrough = _memproj_fallthrough;
0 a61af66fc99e Initial load duke parents: diff changeset	1827
a61af66fc99e Initial load duke parents: diff changeset	1828 // The memory projection from a lock/unlock is RawMem
a61af66fc99e Initial load duke parents: diff changeset	1829 // The input to a Lock is merged memory, so extract its RawMem input
a61af66fc99e Initial load duke parents: diff changeset	1830 // (unless the MergeMem has been optimized away.)
a61af66fc99e Initial load duke parents: diff changeset	1831 if (alock->is_Lock()) {
3849 f1c12354c3f7 7074017: Introduce MemBarAcquireLock/MemBarReleaseLock nodes for monitor enter/exit code paths roland parents: 3788 diff changeset	1832 // Seach for MemBarAcquireLock node and delete it also.
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1833 MemBarNode* membar = fallthroughproj->unique_ctrl_out()->as_MemBar();
3849 f1c12354c3f7 7074017: Introduce MemBarAcquireLock/MemBarReleaseLock nodes for monitor enter/exit code paths roland parents: 3788 diff changeset	1834 assert(membar != NULL && membar->Opcode() == Op_MemBarAcquireLock, "");
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1835 Node* ctrlproj = membar->proj_out(TypeFunc::Control);
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1836 Node* memproj = membar->proj_out(TypeFunc::Memory);
708 f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	1837 _igvn.replace_node(ctrlproj, fallthroughproj);
f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	1838 _igvn.replace_node(memproj, memproj_fallthrough);
460 424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1839
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1840 // Delete FastLock node also if this Lock node is unique user
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1841 // (a loop peeling may clone a Lock node).
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1842 Node* flock = alock->as_Lock()->fastlock_node();
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1843 if (flock->outcnt() == 1) {
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1844 assert(flock->unique_out() == alock, "sanity");
708 f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	1845 _igvn.replace_node(flock, top());
460 424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	1846 }
0 a61af66fc99e Initial load duke parents: diff changeset	1847 }
a61af66fc99e Initial load duke parents: diff changeset	1848
3849 f1c12354c3f7 7074017: Introduce MemBarAcquireLock/MemBarReleaseLock nodes for monitor enter/exit code paths roland parents: 3788 diff changeset	1849 // Seach for MemBarReleaseLock node and delete it also.
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1850 if (alock->is_Unlock() && ctrl != NULL && ctrl->is_Proj() &&
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1851 ctrl->in(0)->is_MemBar()) {
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1852 MemBarNode* membar = ctrl->in(0)->as_MemBar();
3849 f1c12354c3f7 7074017: Introduce MemBarAcquireLock/MemBarReleaseLock nodes for monitor enter/exit code paths roland parents: 3788 diff changeset	1853 assert(membar->Opcode() == Op_MemBarReleaseLock &&
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1854 mem->is_Proj() && membar == mem->in(0), "");
708 f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	1855 _igvn.replace_node(fallthroughproj, ctrl);
f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	1856 _igvn.replace_node(memproj_fallthrough, mem);
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1857 fallthroughproj = ctrl;
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1858 memproj_fallthrough = mem;
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1859 ctrl = membar->in(TypeFunc::Control);
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1860 mem = membar->in(TypeFunc::Memory);
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1861 }
6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1862
708 f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	1863 _igvn.replace_node(fallthroughproj, ctrl);
f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	1864 _igvn.replace_node(memproj_fallthrough, mem);
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1865 return true;
0 a61af66fc99e Initial load duke parents: diff changeset	1866 }
a61af66fc99e Initial load duke parents: diff changeset	1867
a61af66fc99e Initial load duke parents: diff changeset	1868
a61af66fc99e Initial load duke parents: diff changeset	1869 //------------------------------expand_lock_node----------------------
a61af66fc99e Initial load duke parents: diff changeset	1870 void PhaseMacroExpand::expand_lock_node(LockNode *lock) {
a61af66fc99e Initial load duke parents: diff changeset	1871
a61af66fc99e Initial load duke parents: diff changeset	1872 Node* ctrl = lock->in(TypeFunc::Control);
a61af66fc99e Initial load duke parents: diff changeset	1873 Node* mem = lock->in(TypeFunc::Memory);
a61af66fc99e Initial load duke parents: diff changeset	1874 Node* obj = lock->obj_node();
a61af66fc99e Initial load duke parents: diff changeset	1875 Node* box = lock->box_node();
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	1876 Node* flock = lock->fastlock_node();
0 a61af66fc99e Initial load duke parents: diff changeset	1877
a61af66fc99e Initial load duke parents: diff changeset	1878 // Make the merge point
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1879 Node *region;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1880 Node *mem_phi;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1881 Node *slow_path;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1882
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1883 if (UseOptoBiasInlining) {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1884 /*
605 98cb887364d3 6810672: Comment typos twisti parents: 601 diff changeset	1885 * See the full description in MacroAssembler::biased_locking_enter().
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1886 *
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1887 * if( (mark_word & biased_lock_mask) == biased_lock_pattern ) {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1888 * // The object is biased.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1889 * proto_node = klass->prototype_header;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1890 * o_node = thread \| proto_node;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1891 * x_node = o_node ^ mark_word;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1892 * if( (x_node & ~age_mask) == 0 ) { // Biased to the current thread ?
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1893 * // Done.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1894 * } else {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1895 * if( (x_node & biased_lock_mask) != 0 ) {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1896 * // The klass's prototype header is no longer biased.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1897 * cas(&mark_word, mark_word, proto_node)
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1898 * goto cas_lock;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1899 * } else {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1900 * // The klass's prototype header is still biased.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1901 * if( (x_node & epoch_mask) != 0 ) { // Expired epoch?
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1902 * old = mark_word;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1903 * new = o_node;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1904 * } else {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1905 * // Different thread or anonymous biased.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1906 * old = mark_word & (epoch_mask \| age_mask \| biased_lock_mask);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1907 * new = thread \| old;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1908 * }
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1909 * // Try to rebias.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1910 * if( cas(&mark_word, old, new) == 0 ) {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1911 * // Done.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1912 * } else {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1913 * goto slow_path; // Failed.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1914 * }
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1915 * }
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1916 * }
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1917 * } else {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1918 * // The object is not biased.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1919 * cas_lock:
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1920 * if( FastLock(obj) == 0 ) {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1921 * // Done.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1922 * } else {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1923 * slow_path:
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1924 * OptoRuntime::complete_monitor_locking_Java(obj);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1925 * }
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1926 * }
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1927 */
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1928
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1929 region = new (C, 5) RegionNode(5);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1930 // create a Phi for the memory state
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1931 mem_phi = new (C, 5) PhiNode( region, Type::MEMORY, TypeRawPtr::BOTTOM);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1932
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1933 Node* fast_lock_region = new (C, 3) RegionNode(3);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1934 Node* fast_lock_mem_phi = new (C, 3) PhiNode( fast_lock_region, Type::MEMORY, TypeRawPtr::BOTTOM);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1935
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1936 // First, check mark word for the biased lock pattern.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1937 Node* mark_node = make_load(ctrl, mem, obj, oopDesc::mark_offset_in_bytes(), TypeX_X, TypeX_X->basic_type());
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1938
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1939 // Get fast path - mark word has the biased lock pattern.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1940 ctrl = opt_bits_test(ctrl, fast_lock_region, 1, mark_node,
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1941 markOopDesc::biased_lock_mask_in_place,
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1942 markOopDesc::biased_lock_pattern, true);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1943 // fast_lock_region->in(1) is set to slow path.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1944 fast_lock_mem_phi->init_req(1, mem);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1945
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1946 // Now check that the lock is biased to the current thread and has
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1947 // the same epoch and bias as Klass::_prototype_header.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1948
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1949 // Special-case a fresh allocation to avoid building nodes:
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1950 Node* klass_node = AllocateNode::Ideal_klass(obj, &_igvn);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1951 if (klass_node == NULL) {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1952 Node* k_adr = basic_plus_adr(obj, oopDesc::klass_offset_in_bytes());
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1953 klass_node = transform_later( LoadKlassNode::make(_igvn, mem, k_adr, _igvn.type(k_adr)->is_ptr()) );
490 dd70dd4c91de 6782820: Server VM fails with "unhandled implicit exception in compiled code" kvn parents: 460 diff changeset	1954 #ifdef _LP64
dd70dd4c91de 6782820: Server VM fails with "unhandled implicit exception in compiled code" kvn parents: 460 diff changeset	1955 if (UseCompressedOops && klass_node->is_DecodeN()) {
dd70dd4c91de 6782820: Server VM fails with "unhandled implicit exception in compiled code" kvn parents: 460 diff changeset	1956 assert(klass_node->in(1)->Opcode() == Op_LoadNKlass, "sanity");
dd70dd4c91de 6782820: Server VM fails with "unhandled implicit exception in compiled code" kvn parents: 460 diff changeset	1957 klass_node->in(1)->init_req(0, ctrl);
dd70dd4c91de 6782820: Server VM fails with "unhandled implicit exception in compiled code" kvn parents: 460 diff changeset	1958 } else
dd70dd4c91de 6782820: Server VM fails with "unhandled implicit exception in compiled code" kvn parents: 460 diff changeset	1959 #endif
dd70dd4c91de 6782820: Server VM fails with "unhandled implicit exception in compiled code" kvn parents: 460 diff changeset	1960 klass_node->init_req(0, ctrl);
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1961 }
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1962 Node *proto_node = make_load(ctrl, mem, klass_node, Klass::prototype_header_offset_in_bytes() + sizeof(oopDesc), TypeX_X, TypeX_X->basic_type());
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1963
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1964 Node* thread = transform_later(new (C, 1) ThreadLocalNode());
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1965 Node* cast_thread = transform_later(new (C, 2) CastP2XNode(ctrl, thread));
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1966 Node* o_node = transform_later(new (C, 3) OrXNode(cast_thread, proto_node));
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1967 Node* x_node = transform_later(new (C, 3) XorXNode(o_node, mark_node));
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1968
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1969 // Get slow path - mark word does NOT match the value.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1970 Node* not_biased_ctrl = opt_bits_test(ctrl, region, 3, x_node,
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1971 (~markOopDesc::age_mask_in_place), 0);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1972 // region->in(3) is set to fast path - the object is biased to the current thread.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1973 mem_phi->init_req(3, mem);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1974
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1975
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1976 // Mark word does NOT match the value (thread \| Klass::_prototype_header).
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1977
0 a61af66fc99e Initial load duke parents: diff changeset	1978
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1979 // First, check biased pattern.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1980 // Get fast path - _prototype_header has the same biased lock pattern.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1981 ctrl = opt_bits_test(not_biased_ctrl, fast_lock_region, 2, x_node,
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1982 markOopDesc::biased_lock_mask_in_place, 0, true);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1983
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1984 not_biased_ctrl = fast_lock_region->in(2); // Slow path
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1985 // fast_lock_region->in(2) - the prototype header is no longer biased
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1986 // and we have to revoke the bias on this object.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1987 // We are going to try to reset the mark of this object to the prototype
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1988 // value and fall through to the CAS-based locking scheme.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1989 Node* adr = basic_plus_adr(obj, oopDesc::mark_offset_in_bytes());
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1990 Node* cas = new (C, 5) StoreXConditionalNode(not_biased_ctrl, mem, adr,
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1991 proto_node, mark_node);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1992 transform_later(cas);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1993 Node* proj = transform_later( new (C, 1) SCMemProjNode(cas));
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1994 fast_lock_mem_phi->init_req(2, proj);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1995
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1996
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1997 // Second, check epoch bits.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1998 Node* rebiased_region = new (C, 3) RegionNode(3);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	1999 Node* old_phi = new (C, 3) PhiNode( rebiased_region, TypeX_X);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2000 Node* new_phi = new (C, 3) PhiNode( rebiased_region, TypeX_X);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2001
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2002 // Get slow path - mark word does NOT match epoch bits.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2003 Node* epoch_ctrl = opt_bits_test(ctrl, rebiased_region, 1, x_node,
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2004 markOopDesc::epoch_mask_in_place, 0);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2005 // The epoch of the current bias is not valid, attempt to rebias the object
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2006 // toward the current thread.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2007 rebiased_region->init_req(2, epoch_ctrl);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2008 old_phi->init_req(2, mark_node);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2009 new_phi->init_req(2, o_node);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2010
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2011 // rebiased_region->in(1) is set to fast path.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2012 // The epoch of the current bias is still valid but we know
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2013 // nothing about the owner; it might be set or it might be clear.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2014 Node* cmask = MakeConX(markOopDesc::biased_lock_mask_in_place \|
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2015 markOopDesc::age_mask_in_place \|
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2016 markOopDesc::epoch_mask_in_place);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2017 Node* old = transform_later(new (C, 3) AndXNode(mark_node, cmask));
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2018 cast_thread = transform_later(new (C, 2) CastP2XNode(ctrl, thread));
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2019 Node* new_mark = transform_later(new (C, 3) OrXNode(cast_thread, old));
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2020 old_phi->init_req(1, old);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2021 new_phi->init_req(1, new_mark);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2022
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2023 transform_later(rebiased_region);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2024 transform_later(old_phi);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2025 transform_later(new_phi);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2026
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2027 // Try to acquire the bias of the object using an atomic operation.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2028 // If this fails we will go in to the runtime to revoke the object's bias.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2029 cas = new (C, 5) StoreXConditionalNode(rebiased_region, mem, adr,
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2030 new_phi, old_phi);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2031 transform_later(cas);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2032 proj = transform_later( new (C, 1) SCMemProjNode(cas));
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2033
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2034 // Get slow path - Failed to CAS.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2035 not_biased_ctrl = opt_bits_test(rebiased_region, region, 4, cas, 0, 0);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2036 mem_phi->init_req(4, proj);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2037 // region->in(4) is set to fast path - the object is rebiased to the current thread.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2038
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2039 // Failed to CAS.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2040 slow_path = new (C, 3) RegionNode(3);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2041 Node *slow_mem = new (C, 3) PhiNode( slow_path, Type::MEMORY, TypeRawPtr::BOTTOM);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2042
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2043 slow_path->init_req(1, not_biased_ctrl); // Capture slow-control
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2044 slow_mem->init_req(1, proj);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2045
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2046 // Call CAS-based locking scheme (FastLock node).
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2047
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2048 transform_later(fast_lock_region);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2049 transform_later(fast_lock_mem_phi);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2050
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2051 // Get slow path - FastLock failed to lock the object.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2052 ctrl = opt_bits_test(fast_lock_region, region, 2, flock, 0, 0);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2053 mem_phi->init_req(2, fast_lock_mem_phi);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2054 // region->in(2) is set to fast path - the object is locked to the current thread.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2055
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2056 slow_path->init_req(2, ctrl); // Capture slow-control
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2057 slow_mem->init_req(2, fast_lock_mem_phi);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2058
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2059 transform_later(slow_path);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2060 transform_later(slow_mem);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2061 // Reset lock's memory edge.
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2062 lock->set_req(TypeFunc::Memory, slow_mem);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2063
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2064 } else {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2065 region = new (C, 3) RegionNode(3);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2066 // create a Phi for the memory state
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2067 mem_phi = new (C, 3) PhiNode( region, Type::MEMORY, TypeRawPtr::BOTTOM);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2068
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2069 // Optimize test; set region slot 2
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2070 slow_path = opt_bits_test(ctrl, region, 2, flock, 0, 0);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2071 mem_phi->init_req(2, mem);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2072 }
0 a61af66fc99e Initial load duke parents: diff changeset	2073
a61af66fc99e Initial load duke parents: diff changeset	2074 // Make slow path call
a61af66fc99e Initial load duke parents: diff changeset	2075 CallNode call = make_slow_call( (CallNode ) lock, OptoRuntime::complete_monitor_enter_Type(), OptoRuntime::complete_monitor_locking_Java(), NULL, slow_path, obj, box );
a61af66fc99e Initial load duke parents: diff changeset	2076
a61af66fc99e Initial load duke parents: diff changeset	2077 extract_call_projections(call);
a61af66fc99e Initial load duke parents: diff changeset	2078
a61af66fc99e Initial load duke parents: diff changeset	2079 // Slow path can only throw asynchronous exceptions, which are always
a61af66fc99e Initial load duke parents: diff changeset	2080 // de-opted. So the compiler thinks the slow-call can never throw an
a61af66fc99e Initial load duke parents: diff changeset	2081 // exception. If it DOES throw an exception we would need the debug
a61af66fc99e Initial load duke parents: diff changeset	2082 // info removed first (since if it throws there is no monitor).
a61af66fc99e Initial load duke parents: diff changeset	2083 assert ( _ioproj_fallthrough == NULL && _ioproj_catchall == NULL &&
a61af66fc99e Initial load duke parents: diff changeset	2084 _memproj_catchall == NULL && _catchallcatchproj == NULL, "Unexpected projection from Lock");
a61af66fc99e Initial load duke parents: diff changeset	2085
a61af66fc99e Initial load duke parents: diff changeset	2086 // Capture slow path
a61af66fc99e Initial load duke parents: diff changeset	2087 // disconnect fall-through projection from call and create a new one
a61af66fc99e Initial load duke parents: diff changeset	2088 // hook up users of fall-through projection to region
a61af66fc99e Initial load duke parents: diff changeset	2089 Node *slow_ctrl = _fallthroughproj->clone();
a61af66fc99e Initial load duke parents: diff changeset	2090 transform_later(slow_ctrl);
a61af66fc99e Initial load duke parents: diff changeset	2091 _igvn.hash_delete(_fallthroughproj);
a61af66fc99e Initial load duke parents: diff changeset	2092 _fallthroughproj->disconnect_inputs(NULL);
a61af66fc99e Initial load duke parents: diff changeset	2093 region->init_req(1, slow_ctrl);
a61af66fc99e Initial load duke parents: diff changeset	2094 // region inputs are now complete
a61af66fc99e Initial load duke parents: diff changeset	2095 transform_later(region);
708 f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	2096 _igvn.replace_node(_fallthroughproj, region);
0 a61af66fc99e Initial load duke parents: diff changeset	2097
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2098 Node *memproj = transform_later( new(C, 1) ProjNode(call, TypeFunc::Memory) );
0 a61af66fc99e Initial load duke parents: diff changeset	2099 mem_phi->init_req(1, memproj );
a61af66fc99e Initial load duke parents: diff changeset	2100 transform_later(mem_phi);
708 f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	2101 _igvn.replace_node(_memproj_fallthrough, mem_phi);
0 a61af66fc99e Initial load duke parents: diff changeset	2102 }
a61af66fc99e Initial load duke parents: diff changeset	2103
a61af66fc99e Initial load duke parents: diff changeset	2104 //------------------------------expand_unlock_node----------------------
a61af66fc99e Initial load duke parents: diff changeset	2105 void PhaseMacroExpand::expand_unlock_node(UnlockNode *unlock) {
a61af66fc99e Initial load duke parents: diff changeset	2106
66 6dbf1a175d6b 6672848: (Escape Analysis) improve lock elimination with EA kvn parents: 63 diff changeset	2107 Node* ctrl = unlock->in(TypeFunc::Control);
0 a61af66fc99e Initial load duke parents: diff changeset	2108 Node* mem = unlock->in(TypeFunc::Memory);
a61af66fc99e Initial load duke parents: diff changeset	2109 Node* obj = unlock->obj_node();
a61af66fc99e Initial load duke parents: diff changeset	2110 Node* box = unlock->box_node();
a61af66fc99e Initial load duke parents: diff changeset	2111
a61af66fc99e Initial load duke parents: diff changeset	2112 // No need for a null check on unlock
a61af66fc99e Initial load duke parents: diff changeset	2113
a61af66fc99e Initial load duke parents: diff changeset	2114 // Make the merge point
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2115 Node *region;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2116 Node *mem_phi;
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2117
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2118 if (UseOptoBiasInlining) {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2119 // Check for biased locking unlock case, which is a no-op.
605 98cb887364d3 6810672: Comment typos twisti parents: 601 diff changeset	2120 // See the full description in MacroAssembler::biased_locking_exit().
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2121 region = new (C, 4) RegionNode(4);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2122 // create a Phi for the memory state
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2123 mem_phi = new (C, 4) PhiNode( region, Type::MEMORY, TypeRawPtr::BOTTOM);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2124 mem_phi->init_req(3, mem);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2125
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2126 Node* mark_node = make_load(ctrl, mem, obj, oopDesc::mark_offset_in_bytes(), TypeX_X, TypeX_X->basic_type());
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2127 ctrl = opt_bits_test(ctrl, region, 3, mark_node,
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2128 markOopDesc::biased_lock_mask_in_place,
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2129 markOopDesc::biased_lock_pattern);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2130 } else {
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2131 region = new (C, 3) RegionNode(3);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2132 // create a Phi for the memory state
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2133 mem_phi = new (C, 3) PhiNode( region, Type::MEMORY, TypeRawPtr::BOTTOM);
a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2134 }
0 a61af66fc99e Initial load duke parents: diff changeset	2135
a61af66fc99e Initial load duke parents: diff changeset	2136 FastUnlockNode *funlock = new (C, 3) FastUnlockNode( ctrl, obj, box );
a61af66fc99e Initial load duke parents: diff changeset	2137 funlock = transform_later( funlock )->as_FastUnlock();
a61af66fc99e Initial load duke parents: diff changeset	2138 // Optimize test; set region slot 2
420 a1980da045cc 6462850: generate biased locking code in C2 ideal graph kvn parents: 362 diff changeset	2139 Node *slow_path = opt_bits_test(ctrl, region, 2, funlock, 0, 0);
0 a61af66fc99e Initial load duke parents: diff changeset	2140
a61af66fc99e Initial load duke parents: diff changeset	2141 CallNode call = make_slow_call( (CallNode ) unlock, OptoRuntime::complete_monitor_exit_Type(), CAST_FROM_FN_PTR(address, SharedRuntime::complete_monitor_unlocking_C), "complete_monitor_unlocking_C", slow_path, obj, box );
a61af66fc99e Initial load duke parents: diff changeset	2142
a61af66fc99e Initial load duke parents: diff changeset	2143 extract_call_projections(call);
a61af66fc99e Initial load duke parents: diff changeset	2144
a61af66fc99e Initial load duke parents: diff changeset	2145 assert ( _ioproj_fallthrough == NULL && _ioproj_catchall == NULL &&
a61af66fc99e Initial load duke parents: diff changeset	2146 _memproj_catchall == NULL && _catchallcatchproj == NULL, "Unexpected projection from Lock");
a61af66fc99e Initial load duke parents: diff changeset	2147
a61af66fc99e Initial load duke parents: diff changeset	2148 // No exceptions for unlocking
a61af66fc99e Initial load duke parents: diff changeset	2149 // Capture slow path
a61af66fc99e Initial load duke parents: diff changeset	2150 // disconnect fall-through projection from call and create a new one
a61af66fc99e Initial load duke parents: diff changeset	2151 // hook up users of fall-through projection to region
a61af66fc99e Initial load duke parents: diff changeset	2152 Node *slow_ctrl = _fallthroughproj->clone();
a61af66fc99e Initial load duke parents: diff changeset	2153 transform_later(slow_ctrl);
a61af66fc99e Initial load duke parents: diff changeset	2154 _igvn.hash_delete(_fallthroughproj);
a61af66fc99e Initial load duke parents: diff changeset	2155 _fallthroughproj->disconnect_inputs(NULL);
a61af66fc99e Initial load duke parents: diff changeset	2156 region->init_req(1, slow_ctrl);
a61af66fc99e Initial load duke parents: diff changeset	2157 // region inputs are now complete
a61af66fc99e Initial load duke parents: diff changeset	2158 transform_later(region);
708 f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	2159 _igvn.replace_node(_fallthroughproj, region);
0 a61af66fc99e Initial load duke parents: diff changeset	2160
a61af66fc99e Initial load duke parents: diff changeset	2161 Node *memproj = transform_later( new(C, 1) ProjNode(call, TypeFunc::Memory) );
a61af66fc99e Initial load duke parents: diff changeset	2162 mem_phi->init_req(1, memproj );
a61af66fc99e Initial load duke parents: diff changeset	2163 mem_phi->init_req(2, mem);
a61af66fc99e Initial load duke parents: diff changeset	2164 transform_later(mem_phi);
708 f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	2165 _igvn.replace_node(_memproj_fallthrough, mem_phi);
0 a61af66fc99e Initial load duke parents: diff changeset	2166 }
a61af66fc99e Initial load duke parents: diff changeset	2167
a61af66fc99e Initial load duke parents: diff changeset	2168 //------------------------------expand_macro_nodes----------------------
a61af66fc99e Initial load duke parents: diff changeset	2169 // Returns true if a failure occurred.
a61af66fc99e Initial load duke parents: diff changeset	2170 bool PhaseMacroExpand::expand_macro_nodes() {
a61af66fc99e Initial load duke parents: diff changeset	2171 if (C->macro_count() == 0)
a61af66fc99e Initial load duke parents: diff changeset	2172 return false;
460 424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2173 // First, attempt to eliminate locks
3754 642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2174 int cnt = C->macro_count();
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2175 for (int i=0; i < cnt; i++) {
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2176 Node *n = C->macro_node(i);
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2177 if (n->is_AbstractLock()) { // Lock and Unlock nodes
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2178 // Before elimination mark all associated (same box and obj)
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2179 // lock and unlock nodes.
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2180 mark_eliminated_locking_nodes(n->as_AbstractLock());
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2181 }
642c68c75db9 7050280: assert(u->as_Unlock()->is_eliminated()) failed: sanity kvn parents: 3345 diff changeset	2182 }
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2183 bool progress = true;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2184 while (progress) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2185 progress = false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2186 for (int i = C->macro_count(); i > 0; i--) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2187 Node * n = C->macro_node(i-1);
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2188 bool success = false;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2189 debug_only(int old_macro_count = C->macro_count(););
460 424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2190 if (n->is_AbstractLock()) {
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2191 success = eliminate_locking_node(n->as_AbstractLock());
3345 bad7ecd0b6ed 5091921: Sign flip issues in loop optimizer kvn parents: 3282 diff changeset	2192 } else if (n->Opcode() == Op_LoopLimit) {
bad7ecd0b6ed 5091921: Sign flip issues in loop optimizer kvn parents: 3282 diff changeset	2193 // Remove it from macro list and put on IGVN worklist to optimize.
bad7ecd0b6ed 5091921: Sign flip issues in loop optimizer kvn parents: 3282 diff changeset	2194 C->remove_macro_node(n);
bad7ecd0b6ed 5091921: Sign flip issues in loop optimizer kvn parents: 3282 diff changeset	2195 _igvn._worklist.push(n);
bad7ecd0b6ed 5091921: Sign flip issues in loop optimizer kvn parents: 3282 diff changeset	2196 success = true;
460 424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2197 } else if (n->Opcode() == Op_Opaque1 \|\| n->Opcode() == Op_Opaque2) {
708 f2049ae95c3d 6711117: Assertion in 64bit server vm (flat != TypePtr::BOTTOM,"cannot alias-analyze an untyped ptr") kvn parents: 628 diff changeset	2198 _igvn.replace_node(n, n->in(1));
460 424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2199 success = true;
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2200 }
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2201 assert(success == (C->macro_count() < old_macro_count), "elimination reduces macro count");
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2202 progress = progress \|\| success;
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2203 }
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2204 }
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2205 // Next, attempt to eliminate allocations
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2206 progress = true;
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2207 while (progress) {
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2208 progress = false;
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2209 for (int i = C->macro_count(); i > 0; i--) {
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2210 Node * n = C->macro_node(i-1);
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2211 bool success = false;
424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2212 debug_only(int old_macro_count = C->macro_count(););
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2213 switch (n->class_id()) {
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2214 case Node::Class_Allocate:
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2215 case Node::Class_AllocateArray:
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2216 success = eliminate_allocate_node(n->as_Allocate());
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2217 break;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2218 case Node::Class_Lock:
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2219 case Node::Class_Unlock:
460 424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2220 assert(!n->as_AbstractLock()->is_eliminated(), "sanity");
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2221 break;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2222 default:
460 424f9bfe6b96 6775880: EA +DeoptimizeALot: assert(mon_info->owner()->is_locked(),"object must be locked now") kvn parents: 436 diff changeset	2223 assert(false, "unknown node type in macro list");
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2224 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2225 assert(success == (C->macro_count() < old_macro_count), "elimination reduces macro count");
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2226 progress = progress \|\| success;
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2227 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2228 }
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2229 // Make sure expansion will not cause node limit to be exceeded.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2230 // Worst case is a macro node gets expanded into about 50 nodes.
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2231 // Allow 50% more for optimization.
0 a61af66fc99e Initial load duke parents: diff changeset	2232 if (C->check_node_count(C->macro_count() * 75, "out of nodes before macro expansion" ) )
a61af66fc99e Initial load duke parents: diff changeset	2233 return true;
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2234
0 a61af66fc99e Initial load duke parents: diff changeset	2235 // expand "macro" nodes
a61af66fc99e Initial load duke parents: diff changeset	2236 // nodes are removed from the macro list as they are processed
a61af66fc99e Initial load duke parents: diff changeset	2237 while (C->macro_count() > 0) {
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2238 int macro_count = C->macro_count();
a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2239 Node * n = C->macro_node(macro_count-1);
0 a61af66fc99e Initial load duke parents: diff changeset	2240 assert(n->is_macro(), "only macro nodes expected here");
a61af66fc99e Initial load duke parents: diff changeset	2241 if (_igvn.type(n) == Type::TOP \|\| n->in(0)->is_top() ) {
a61af66fc99e Initial load duke parents: diff changeset	2242 // node is unreachable, so don't try to expand it
a61af66fc99e Initial load duke parents: diff changeset	2243 C->remove_macro_node(n);
a61af66fc99e Initial load duke parents: diff changeset	2244 continue;
a61af66fc99e Initial load duke parents: diff changeset	2245 }
a61af66fc99e Initial load duke parents: diff changeset	2246 switch (n->class_id()) {
a61af66fc99e Initial load duke parents: diff changeset	2247 case Node::Class_Allocate:
a61af66fc99e Initial load duke parents: diff changeset	2248 expand_allocate(n->as_Allocate());
a61af66fc99e Initial load duke parents: diff changeset	2249 break;
a61af66fc99e Initial load duke parents: diff changeset	2250 case Node::Class_AllocateArray:
a61af66fc99e Initial load duke parents: diff changeset	2251 expand_allocate_array(n->as_AllocateArray());
a61af66fc99e Initial load duke parents: diff changeset	2252 break;
a61af66fc99e Initial load duke parents: diff changeset	2253 case Node::Class_Lock:
a61af66fc99e Initial load duke parents: diff changeset	2254 expand_lock_node(n->as_Lock());
a61af66fc99e Initial load duke parents: diff changeset	2255 break;
a61af66fc99e Initial load duke parents: diff changeset	2256 case Node::Class_Unlock:
a61af66fc99e Initial load duke parents: diff changeset	2257 expand_unlock_node(n->as_Unlock());
a61af66fc99e Initial load duke parents: diff changeset	2258 break;
a61af66fc99e Initial load duke parents: diff changeset	2259 default:
a61af66fc99e Initial load duke parents: diff changeset	2260 assert(false, "unknown node type in macro list");
a61af66fc99e Initial load duke parents: diff changeset	2261 }
73 a8880a78d355 6259129: (Escape Analysis) scalar replacement for not escaping objects kvn parents: 66 diff changeset	2262 assert(C->macro_count() < macro_count, "must have deleted a node from macro list");
0 a61af66fc99e Initial load duke parents: diff changeset	2263 if (C->failing()) return true;
a61af66fc99e Initial load duke parents: diff changeset	2264 }
113 ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 73 diff changeset	2265
ba764ed4b6f2 6420645: Create a vm that uses compressed oops for up to 32gb heapsizes coleenp parents: 73 diff changeset	2266 _igvn.set_delay_transform(false);
0 a61af66fc99e Initial load duke parents: diff changeset	2267 _igvn.optimize();
a61af66fc99e Initial load duke parents: diff changeset	2268 return false;
a61af66fc99e Initial load duke parents: diff changeset	2269 }

Mercurial > hg > graal-compiler

annotate src/share/vm/opto/macro.cpp @ 3992:d1bdeef3e3e2