Running ./CMSSW.sh with 4 positional parameters: 1 /store/mc/Summer08/QCDDiJetPt15to20/GEN-SIM-RECO/IDEAL_V9_v2 /0004/1C460A70-3C9E-DD11-8C5B-00192165CCB4.root -1 0 Today is Mon Apr 20 22:41:30 CEST 2009 Job submitted on host grid277.lal.in2p3.fr Linux grid277.lal.in2p3.fr 2.6.9-67.0.4.ELlargesmp #1 SMP Thu Jan 31 16:19:47 CST 2008 x86_64 x86_64 x86_64 GNU /Linux >>> current directory (RUNTIME_AREA): /var/spool/pbs/tmpdir/1131220.grid10.lal.in2p3.fr/https_3a_2f_2fwms218.ce rn.ch_3a9000_2fD77O5zgqtRj3UKxnAkkYgA >>> current directory content: total 5276 -rw------- 1 cms157 cms 5887 Apr 20 22:41 .BrokerInfo -rw------- 1 cms157 cms 0 Apr 20 22:41 CMSSW_1.stderr -rw------- 1 cms157 cms 544 Apr 20 22:41 CMSSW_1.stdout -rwx------ 1 cms157 cms 15502 Apr 20 22:41 CMSSW.sh -rw------- 1 cms157 cms 5361429 Apr 20 22:41 default.tgz -rw------- 1 cms157 cms 0 Apr 20 22:41 https_3a_2f_2fwms218.cern.ch_3a9000_2fD77O5zgqtRj3UKxnAkkYgA.outp ut -rw------- 1 cms157 cms 0 Apr 20 22:41 tmp.FzARG20207 >>> current user: uid=4183(cms157) gid=4005(cms) groups=4005(cms) >>> voms proxy information: subject : /DC=org/DC=doegrids/OU=People/CN=Xin Shi 851720/CN=proxy/CN=proxy/CN=limited proxy issuer : /DC=org/DC=doegrids/OU=People/CN=Xin Shi 851720/CN=proxy/CN=proxy identity : /DC=org/DC=doegrids/OU=People/CN=Xin Shi 851720/CN=proxy/CN=proxy type : limited proxy strength : 512 bits path : /vohome/cms/cms157/.globus/job/grid10.lal.in2p3.fr/27857.1240252605/x509_up timeleft : 189:52:13 === VO cms extension information === VO : cms subject : /DC=org/DC=doegrids/OU=People/CN=Xin Shi 851720 issuer : /DC=ch/DC=cern/OU=computers/CN=lcg-voms.cern.ch attribute : /cms/Role=NULL/Capability=NULL attribute : /cms/uscms/Role=NULL/Capability=NULL timeleft : 189:53:01 uri : lcg-voms.cern.ch:15002 >>> tar xzvf /var/spool/pbs/tmpdir/1131220.grid10.lal.in2p3.fr/https_3a_2f_2fwms218.cern.ch_3a9000_2fD77O5zgqtR j3UKxnAkkYgA/default.tgz : Successful untar >>> Include /var/spool/pbs/tmpdir/1131220.grid10.lal.in2p3.fr/https_3a_2f_2fwms218.cern.ch_3a9000_2fD77O5zgqtRj 3UKxnAkkYgA in PYTHONPATH: PYTHONPATH=/var/spool/pbs/tmpdir/1131220.grid10.lal.in2p3.fr/https_3a_2f_2fwms218.cern.ch_3a9000_2fD77O5zgqtRj3 UKxnAkkYgA/:/opt/lcg/lib:/opt/lcg/lib/python:/opt/edg/lib:/opt/edg/lib/python strip arguments out_files_1 >>> list of expected files on output sandbox output files: ana_qcd_1.root crab_fjr_1.xml CMSSW_1.stdout CMSSW_1.stderr >>> GridFlavour discovery: GridFlavour=LCG >>> info for dashboard: ***** Cat /var/spool/pbs/tmpdir/1131220.grid10.lal.in2p3.fr/https_3a_2f_2fwms218.cern.ch_3a9000_2fD77O5zgqtRj3U KxnAkkYgA/jobreport.txt ***** MonitorJobID=1_https://wms218.cern.ch:9000/D77O5zgqtRj3UKxnAkkYgA SyncGridJobId=https://wms218.cern.ch:9000/D77O5zgqtRj3UKxnAkkYgA MonitorID=xshi_crab_0_090420_133421_5a3o8f SyncCE=grid10.lal.in2p3.fr:2119/jobmanager-pbs-cms GridFlavour=LCG ***** End Cat jobreport ***** Parameters sent to Dashboard. CloseCEs = grid10.lal.in2p3.fr:2119/jobmanager-pbs-cms CE = grid10.lal.in2p3.fr >>> setup environment >>> setup CMS LCG environment: set SCRAM ARCH and BUILD_ARCH to slc4_ia32_gcc345 ### Sourcing environment... sourcing /swareas/cms/cmsset_default.sh ==> setup cms environment ok >>> specific cmssw setup environment: CMSSW_VERSION = CMSSW_2_2_7 >>> current directory (SOFTWARE_DIR): /var/spool/pbs/tmpdir/1131220.grid10.lal.in2p3.fr/https_3a_2f_2fwms218.ce rn.ch_3a9000_2fD77O5zgqtRj3UKxnAkkYgA/CMSSW_2_2_7 Inputfiles: MaxEvents:<-1> SkipEvents:<0> PSETHASH = 4e8e8d0d99cf4d731881b6aa6f64a94c >>> moving CMSSW software directories in /var/spool/pbs/tmpdir/1131220.grid10.lal.in2p3.fr/https_3a_2f_2fwms218 .cern.ch_3a9000_2fD77O5zgqtRj3UKxnAkkYgA/CMSSW_2_2_7 >>> Include /var/spool/pbs/tmpdir/1131220.grid10.lal.in2p3.fr/https_3a_2f_2fwms218.cern.ch_3a9000_2fD77O5zgqtRj 3UKxnAkkYgA in PYTHONPATH: PYTHONPATH=/var/spool/pbs/tmpdir/1131220.grid10.lal.in2p3.fr/https_3a_2f_2fwms218.cern.ch_3a9000_2fD77O5zgqtRj3 UKxnAkkYgA/:/var/spool/pbs/tmpdir/1131220.grid10.lal.in2p3.fr/https_3a_2f_2fwms218.cern.ch_3a9000_2fD77O5zgqtRj 3UKxnAkkYgA/CMSSW_2_2_7/lib/slc4_ia32_gcc345:/var/spool/pbs/tmpdir/1131220.grid10.lal.in2p3.fr/https_3a_2f_2fwm s218.cern.ch_3a9000_2fD77O5zgqtRj3UKxnAkkYgA/CMSSW_2_2_7/python:/grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmss w/CMSSW_2_2_7/lib/slc4_ia32_gcc345:/grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/python:/swareas /cms/slc4_ia32_gcc345/cms/pool/POOL_2_7_3-cms28/slc4_ia32_gcc345/bin:/swareas/cms/slc4_ia32_gcc345/cms/pool/POO L_2_7_3-cms28/slc4_ia32_gcc345/lib:/swareas/cms/slc4_ia32_gcc345/cms/coral/CORAL_1_9_7-cms11/slc4_ia32_gcc345/p ython:/swareas/cms/slc4_ia32_gcc345/cms/coral/CORAL_1_9_7-cms11/slc4_ia32_gcc345/lib:/swareas/cms/slc4_ia32_gcc 345/cms/seal/SEAL_1_9_4-cms27/slc4_ia32_gcc345/lib:/swareas/cms/slc4_ia32_gcc345/external/python-ldap/2.3.5-cms /lib/python2.4/site-packages:/swareas/cms/slc4_ia32_gcc345/external/elementtree/1.2.6-cms/share/lib/python2.4/s ite-packages:/swareas/cms/slc4_ia32_gcc345/lcg/root/5.18.00a-cms17/lib:/var/spool/pbs/tmpdir/1131220.grid10.lal .in2p3.fr/https_3a_2f_2fwms218.cern.ch_3a9000_2fD77O5zgqtRj3UKxnAkkYgA/:/opt/lcg/lib:/opt/lcg/lib/python:/opt/e dg/lib:/opt/edg/lib/python /var/spool/pbs/tmpdir/1131220.grid10.lal.in2p3.fr/https_3a_2f_2fwms218.cern.ch_3a9000_2fD77O5zgqtRj3UKxnAkkYgA/ writeCfg.py pset.py pset.py Importing .py file Setting parameters >>> Executable cmsRun /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/bin/slc4_ia32_gcc345/cmsRun ok executable found >>> info for dashboard: ***** Cat /var/spool/pbs/tmpdir/1131220.grid10.lal.in2p3.fr/https_3a_2f_2fwms218.cern.ch_3a9000_2fD77O5zgqtRj3U KxnAkkYgA/jobreport.txt ***** MonitorJobID=1_https://wms218.cern.ch:9000/D77O5zgqtRj3UKxnAkkYgA MonitorID=xshi_crab_0_090420_133421_5a3o8f ExeStart=cmsRun ***** End Cat jobreport ***** Parameters sent to Dashboard. >>> cmsRun started at Mon Apr 20 22:41:41 CEST 2009 (no debugging symbols found) Using host libthread_db library "/lib64/tls/libthread_db.so.1". Attaching to program: /proc/20329/exe, process 20329 (no debugging symbols found)...done. (no debugging symbols found)...done. (no debugging symbols found)...done. (no debugging symbols found)...done. (no debugging symbols found)...done. (no debugging symbols found)...done. (no debugging symbols found)...done. (no debugging symbols found)...done. (no debugging symbols found)...done. (no debugging symbols found)...done. (no debugging symbols found)...done. (no debugging symbols found)...done. [Thread debugging using libthread_db enabled] [New Thread 4125563104 (LWP 20329)] [New Thread 3936406432 (LWP 20336)] [New Thread 3946896288 (LWP 20335)] [New Thread 4125072288 (LWP 20330)] 0xffffe410 in __kernel_vsyscall () Thread 4 (Thread 4125072288 (LWP 20330)): #0 0xffffe410 in __kernel_vsyscall () #1 0x00485cf6 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/tls/libpthread.so.0 #2 0xf60e3584 in boost::detail::condition_impl::do_wait () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/external/slc4_ia32_gcc345/lib/libboost_thr ead-gcc34-mt-1_34_1.so.1.34.1 #3 0xf79ac0d8 in edm::SingleConsumerQ::getConsumerBuffer () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/lib/slc4_ia32_gcc345/libFWCoreUtilities.so #4 0xf7a0eb3e in edm::MessageLoggerQ::consume () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/lib/slc4_ia32_gcc345/libFWCoreMessageLogge r.so #5 0xf5e51104 in edm::service::MessageLoggerScribe::run () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/lib/slc4_ia32_gcc345/libFWCoreMessageServi ce.so #6 0xf5e553c1 in (anonymous namespace)::runMessageLoggerScribe () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/lib/slc4_ia32_gcc345/libFWCoreMessageServi ce.so #7 0xf5e558ab in boost::detail::function::void_function_invoker0::invoke () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/lib/slc4_ia32_gcc345/libFWCoreMessageServi ce.so #8 0xf7848aa9 in boost::function0 >::operator() () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/external/slc4_ia32_gcc345/lib/libboost_pyt hon-gcc34-mt-1_34_1.so.1.34.1 #9 0xf60e5c2c in boost::thread_group::join_all () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/external/slc4_ia32_gcc345/lib/libboost_thr ead-gcc34-mt-1_34_1.so.1.34.1 #10 0x004833cc in start_thread () from /lib/tls/libpthread.so.0 #11 0x0031519e in clone () from /lib/tls/libc.so.6 Thread 3 (Thread 3946896288 (LWP 20335)): #0 0xffffe410 in __kernel_vsyscall () #1 0x00485cf6 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/tls/libpthread.so.0 #2 0xeb58f18c in globus_cond_wait (cv=0xeb59bbe0, mut=0xeb59bbac) at globus_thread_pthreads.c:939 #3 0xeb578015 in globus_l_callback_thread_signal_poll (user_arg=0x0) at globus_callback_threads.c:2825 #4 0xeb58e820 in thread_starter (temparg=0xdf53860) at globus_thread_pthreads.c:508 #5 0x004833cc in start_thread () from /lib/tls/libpthread.so.0 #6 0x0031519e in clone () from /lib/tls/libc.so.6 Thread 2 (Thread 3936406432 (LWP 20336)): #0 0xffffe410 in __kernel_vsyscall () #1 0x00485cf6 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/tls/libpthread.so.0 #2 0xeb58f18c in globus_cond_wait (cv=0xeb59bb5c, mut=0xeb59bb44) at globus_thread_pthreads.c:939 #3 0xeb577715 in globus_l_callback_thread_poll (user_arg=0xeb59bb20) at globus_callback_threads.c:2432 #4 0xeb58f8d2 in globus_l_thread_pool_thread_start (user_arg=0xdf554a8) at globus_thread_pool.c:217 #5 0xeb58e820 in thread_starter (temparg=0xdf53870) at globus_thread_pthreads.c:508 #6 0x004833cc in start_thread () from /lib/tls/libpthread.so.0 #7 0x0031519e in clone () from /lib/tls/libc.so.6 Thread 1 (Thread 4125563104 (LWP 20329)): #0 0xffffe410 in __kernel_vsyscall () #1 0x002d6d4b in __waitpid_nocancel () from /lib/tls/libc.so.6 #2 0x002807a9 in do_system () from /lib/tls/libc.so.6 #3 0x00280b21 in system () from /lib/tls/libc.so.6 #4 0x0048998d in system () from /lib/tls/libpthread.so.0 #5 0xf6763d4d in TUnixSystem::Exec () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/external/slc4_ia32_gcc345/lib/libCore.so #6 0xf676a3d0 in TUnixSystem::StackTrace () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/external/slc4_ia32_gcc345/lib/libCore.so #7 0xf6766959 in TUnixSystem::DispatchSignals () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/external/slc4_ia32_gcc345/lib/libCore.so #8 0xf6766a03 in SigHandler () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/external/slc4_ia32_gcc345/lib/libCore.so #9 0xf6765aae in sighandler () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/external/slc4_ia32_gcc345/lib/libCore.so #10 #11 0xe35d1453 in MHTEventSelector::select () from /var/spool/pbs/tmpdir/1131220.grid10.lal.in2p3.fr/https_3a_2f_2fwms218.cern.ch_3a9000_2fD77O5zgqtRj3UKx nAkkYgA/CMSSW_2_2_7/lib/slc4_ia32_gcc345/pluginSusyAnalysisEventSelectorPlugins.so #12 0xe360a935 in SelectorSequence::decisions () from /var/spool/pbs/tmpdir/1131220.grid10.lal.in2p3.fr/https_3a_2f_2fwms218.cern.ch_3a9000_2fD77O5zgqtRj3UKx nAkkYgA/CMSSW_2_2_7/lib/slc4_ia32_gcc345/libSusyAnalysisEventSelector.so #13 0xe3620d00 in SusyAllHadronicAnalysis::analyze () from /var/spool/pbs/tmpdir/1131220.grid10.lal.in2p3.fr/https_3a_2f_2fwms218.cern.ch_3a9000_2fD77O5zgqtRj3UKx nAkkYgA/CMSSW_2_2_7/lib/slc4_ia32_gcc345/pluginSusyAnalysisAllHadronic.so #14 0xf7e73371 in edm::EDAnalyzer::doEvent () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/lib/slc4_ia32_gcc345/libFWCoreFramework.so #15 0xf46bd529 in edm::WorkerT::implDoBegin () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/lib/slc4_ia32_gcc345/pluginPhysicsToolsUti lAlgos_plugins.so #16 0xf7ecfe50 in edm::Worker::doWork > () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/lib/slc4_ia32_gcc345/libFWCoreFramework.so #17 0xf7edcbab in edm::WorkerInPath::runWorker > () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/lib/slc4_ia32_gcc345/libFWCoreFramework.so #18 0xf7eddd0a in edm::Path::processOneOccurrence > () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/lib/slc4_ia32_gcc345/libFWCoreFramework.so #19 0xf7ede796 in std::for_each<__gnu_cxx::__normal_iterator > >, edm::ProcessOneOccurrence > > () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/lib/slc4_ia32_gcc345/libFWCoreFramework.so #20 0xf7ede810 in edm::Schedule::runTriggerPaths > () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/lib/slc4_ia32_gcc345/libFWCoreFramework.so #21 0xf7edeb6e in edm::Schedule::processOneOccurrence > () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/lib/slc4_ia32_gcc345/libFWCoreFramework.so #22 0xf7eb8154 in edm::EventProcessor::processEvent () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/lib/slc4_ia32_gcc345/libFWCoreFramework.so #23 0xf7e81bbb in statemachine::HandleEvent::readAndProcessEvent () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/lib/slc4_ia32_gcc345/libFWCoreFramework.so #24 0xf7e81fc2 in statemachine::HandleEvent::HandleEvent () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/lib/slc4_ia32_gcc345/libFWCoreFramework.so #25 0xf7e9c9ab in boost::statechart::simple_state, (boost::state chart::history_mode)0>::react_impl () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/lib/slc4_ia32_gcc345/libFWCoreFramework.so #26 0xf7edd622 in boost::statechart::state_machine, boost::statechart::null_exception_translator>::send_event () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/lib/slc4_ia32_gcc345/libFWCoreFramework.so #27 0xf7edd9ec in boost::statechart::state_machine, boost::statechart::null_exception_translator>::process_event () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/lib/slc4_ia32_gcc345/libFWCoreFramework.so #28 0xf7ec9430 in edm::EventProcessor::runCommon () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/lib/slc4_ia32_gcc345/libFWCoreFramework.so #29 0xf7eca730 in edm::EventProcessor::runToCompletion () from /grid_mnt/swareas/cms/slc4_ia32_gcc345/cms/cmssw/CMSSW_2_2_7/lib/slc4_ia32_gcc345/libFWCoreFramework.so #30 0x08053314 in main () >>> cmsRun ended at Mon Apr 20 22:43:08 CEST 2009 >>> info for dashboard: ***** Cat /var/spool/pbs/tmpdir/1131220.grid10.lal.in2p3.fr/https_3a_2f_2fwms218.cern.ch_3a9000_2fD77O5zgqtRj3U KxnAkkYgA/jobreport.txt ***** MonitorJobID=1_https://wms218.cern.ch:9000/D77O5zgqtRj3UKxnAkkYgA MonitorID=xshi_crab_0_090420_133421_5a3o8f ExeEnd=cmsRun ***** End Cat jobreport ***** Parameters sent to Dashboard. TIME_EXE = 87 sec >>> Parse FrameworkJobReport crab_fjr.xml >>> crab_fjr.xml contents: /var/spool/pbs/tmpdir/1131220.grid10.lal.in2p3.fr/https_3a_2f_2fwms218.cern.ch_3a9000_2fD77O5zgqtRj 3UKxnAkkYgA/CMSSW_2_2_7/ana_qcd.root Wrong FrameworkJobReport --> does not contain useful info. ExitStatus: 50115 >>> Executable failed 50115 ExeExitCode=50115 EXECUTABLE_EXIT_STATUS = 50115 WARNING: output file ana_qcd_1.root not found! tarring file crab_fjr_1.xml in out_files_1 tarring file CMSSW_1.stdout in out_files_1 tarring file CMSSW_1.stderr in out_files_1 >>> info for dashboard: ***** Cat /var/spool/pbs/tmpdir/1131220.grid10.lal.in2p3.fr/https_3a_2f_2fwms218.cern.ch_3a9000_2fD77O5zgqtRj3U KxnAkkYgA/jobreport.txt ***** MonitorJobID=1_https://wms218.cern.ch:9000/D77O5zgqtRj3UKxnAkkYgA MonitorID=xshi_crab_0_090420_133421_5a3o8f ExeTime=87 ExeExitCode=50115 CrabCpuTime=79.18 2.50 93% CrabWrapperTime=100 CrabStageoutTime=NULL ***** End Cat jobreport ***** Parameters sent to Dashboard. crab_fjr_1.xml CMSSW_1.stdout CMSSW_1.stderr .BrokerInfo Total Output dimension: 5610 WARNING: output files size limit is set to: 55000000 Total Output dimension 5610 is fine. JOB_EXIT_STATUS = 50115 >>> info for dashboard: ***** Cat /var/spool/pbs/tmpdir/1131220.grid10.lal.in2p3.fr/https_3a_2f_2fwms218.cern.ch_3a9000_2fD77O5zgqtRj3U KxnAkkYgA/jobreport.txt ***** MonitorJobID=1_https://wms218.cern.ch:9000/D77O5zgqtRj3UKxnAkkYgA MonitorID=xshi_crab_0_090420_133421_5a3o8f JobExitCode=50115 ***** End Cat jobreport ***** Parameters sent to Dashboard. crab_fjr_1.xml