2019-11-21 16:15:41: xmlchange success <command> ./xmlchange DOUT_S=TRUE  </command>
 ---------------------------------------------------
2019-11-21 16:15:41: xmlchange success <command> ./xmlchange RESUBMIT=25  </command>
 ---------------------------------------------------
2019-11-21 16:15:42: xmlchange success <command> ./xmlchange STOP_N=5  </command>
 ---------------------------------------------------
2019-11-21 16:15:42: xmlchange success <command> ./xmlchange STOP_OPTION=nyears  </command>
 ---------------------------------------------------
2019-11-21 16:15:47: case.setup starting 
 ---------------------------------------------------
2019-11-21 16:15:48: case.setup success 
 ---------------------------------------------------
2019-11-21 16:18:40: xmlchange success <command> ./xmlchange RESUBMIT=0  </command>
 ---------------------------------------------------
2019-11-21 16:26:52: case.run starting 
 ---------------------------------------------------
2019-11-21 16:26:53: case.run error 
ERROR: BUILD_COMPLETE is not true
Please rebuild the model interactively
 ---------------------------------------------------
2019-11-21 16:33:31: case.setup starting 
 ---------------------------------------------------
2019-11-21 16:33:32: build.clean starting 
 ---------------------------------------------------
2019-11-21 16:33:44: build.clean success 
 ---------------------------------------------------
2019-11-21 16:33:44: case.setup success 
 ---------------------------------------------------
2019-11-21 16:36:07: case.build starting 
 ---------------------------------------------------
CESM version is cesm2.1.2-rc.01
Processing externals description file : Externals.cfg
Processing externals description file : Externals_CLM.cfg
Processing externals description file : Externals_POP.cfg
Processing externals description file : Externals_CISM.cfg
Checking status of externals: clm, fates, ptclm, mosart, ww3, cime, cice, pop, cvmix, marbl, cism, source_cism, rtm, cam, 
    ./cime
        clean sandbox, on cime5.6.21
    ./components/cam
        clean sandbox, on cam1/release_tags/cam_cesm2_1_rel_31/components/cam
    ./components/cice
        clean sandbox, on cice5_cesm2_1_1_20190321
    ./components/cism
        clean sandbox, on release-cesm2.0.04
    ./components/cism/source_cism
        clean sandbox, on release-cism2.1.03
    ./components/clm
        clean sandbox, on release-clm5.0.29
    ./components/clm/src/fates
        clean sandbox, on fates_s1.21.0_a7.0.0_br_rev2
    ./components/clm/tools/PTCLM
        clean sandbox, on PTCLM2_180611
    ./components/mosart
        clean sandbox, on release-cesm2.0.03
    ./components/pop
        clean sandbox, on pop2_cesm2_1_rel_n06
    ./components/pop/externals/CVMix
        clean sandbox, on v0.93-beta
    ./components/pop/externals/MARBL
        clean sandbox, on cesm2.1-n00
    ./components/rtm
        clean sandbox, on release-cesm2.0.03
    ./components/ww3
        clean sandbox, on ww3_181001
2019-11-21 16:45:09: case.build success 
 ---------------------------------------------------
2019-11-21 17:12:11: case.run starting 
 ---------------------------------------------------
2019-11-21 17:12:16: model execution starting 
 ---------------------------------------------------
2019-11-21 17:18:49: model execution success 
 ---------------------------------------------------
2019-11-21 17:18:49: case.run error 
ERROR: RUN FAIL: Command 'mpiexec_mpt -p "%g:"  -np 1152  omplace -tm open64  /glade/scratch/cmip6/f.e21.FHIST_SSP126.f09_f09_mg17.CMIP6-lfmip-pdLC.001/bld/cesm.exe  >> cesm.log.$LID 2>&1 ' failed
See log file for details: /glade/scratch/cmip6/f.e21.FHIST_SSP126.f09_f09_mg17.CMIP6-lfmip-pdLC.001/run/cesm.log.9210960.chadmin1.ib0.cheyenne.ucar.edu.191121-171211
 ---------------------------------------------------
2019-11-25 11:15:18: build.clean starting 
 ---------------------------------------------------
2019-11-25 11:15:24: build.clean success 
 ---------------------------------------------------
2019-11-25 11:15:34: case.build starting 
 ---------------------------------------------------
CESM version is cesm2.1.2-rc.01
Processing externals description file : Externals.cfg
Processing externals description file : Externals_CLM.cfg
Processing externals description file : Externals_POP.cfg
Processing externals description file : Externals_CISM.cfg
Checking status of externals: clm, fates, ptclm, mosart, ww3, cime, cice, pop, cvmix, marbl, cism, source_cism, rtm, cam, 
    ./cime
        clean sandbox, on cime5.6.21
    ./components/cam
        clean sandbox, on cam1/release_tags/cam_cesm2_1_rel_31/components/cam
    ./components/cice
        clean sandbox, on cice5_cesm2_1_1_20190321
    ./components/cism
        clean sandbox, on release-cesm2.0.04
    ./components/cism/source_cism
        clean sandbox, on release-cism2.1.03
    ./components/clm
        clean sandbox, on release-clm5.0.29
    ./components/clm/src/fates
        clean sandbox, on fates_s1.21.0_a7.0.0_br_rev2
    ./components/clm/tools/PTCLM
        clean sandbox, on PTCLM2_180611
    ./components/mosart
        clean sandbox, on release-cesm2.0.03
    ./components/pop
        clean sandbox, on pop2_cesm2_1_rel_n06
    ./components/pop/externals/CVMix
        clean sandbox, on v0.93-beta
    ./components/pop/externals/MARBL
        clean sandbox, on cesm2.1-n00
    ./components/rtm
        clean sandbox, on release-cesm2.0.03
    ./components/ww3
        clean sandbox, on ww3_181001
2019-11-25 11:24:54: case.build success 
 ---------------------------------------------------
2019-11-25 12:31:51: case.run starting 
 ---------------------------------------------------
2019-11-25 12:31:56: model execution starting 
 ---------------------------------------------------
2019-11-25 16:11:33: model execution success 
 ---------------------------------------------------
2019-11-25 16:11:33: case.run success 
 ---------------------------------------------------
2019-11-25 16:11:41: xmlchange success <command> ./xmlchange CONTINUE_RUN=TRUE  </command>
 ---------------------------------------------------
2019-11-25 16:11:50: st_archive starting 
 ---------------------------------------------------
2019-11-25 16:11:58: st_archive success 
 ---------------------------------------------------
2019-11-25 16:14:28: case.run starting 
 ---------------------------------------------------
2019-11-25 16:14:32: model execution starting 
 ---------------------------------------------------
2019-11-25 16:26:10: case.run starting 
 ---------------------------------------------------
2019-11-25 16:26:14: model execution starting 
 ---------------------------------------------------
2019-11-25 20:49:00: case.build starting 
 ---------------------------------------------------
CESM version is cesm2.1.2-rc.01
Processing externals description file : Externals.cfg
Processing externals description file : Externals_CLM.cfg
Processing externals description file : Externals_POP.cfg
Processing externals description file : Externals_CISM.cfg
Checking status of externals: clm, fates, ptclm, mosart, ww3, cime, cice, pop, cvmix, marbl, cism, source_cism, rtm, cam, 
    ./cime
        clean sandbox, on cime5.6.21
    ./components/cam
        clean sandbox, on cam1/release_tags/cam_cesm2_1_rel_31/components/cam
    ./components/cice
        clean sandbox, on cice5_cesm2_1_1_20190321
    ./components/cism
        clean sandbox, on release-cesm2.0.04
    ./components/cism/source_cism
        clean sandbox, on release-cism2.1.03
    ./components/clm
        clean sandbox, on release-clm5.0.29
    ./components/clm/src/fates
        clean sandbox, on fates_s1.21.0_a7.0.0_br_rev2
    ./components/clm/tools/PTCLM
        clean sandbox, on PTCLM2_180611
    ./components/mosart
        clean sandbox, on release-cesm2.0.03
    ./components/pop
        clean sandbox, on pop2_cesm2_1_rel_n06
    ./components/pop/externals/CVMix
        clean sandbox, on v0.93-beta
    ./components/pop/externals/MARBL
        clean sandbox, on cesm2.1-n00
    ./components/rtm
        clean sandbox, on release-cesm2.0.03
    ./components/ww3
        clean sandbox, on ww3_181001
2019-11-25 20:49:48: case.build success 
 ---------------------------------------------------
2019-11-25 21:18:29: case.run starting 
 ---------------------------------------------------
2019-11-25 21:18:34: model execution starting 
 ---------------------------------------------------
2019-11-26 01:09:35: model execution success 
 ---------------------------------------------------
2019-11-26 01:09:35: case.run success 
 ---------------------------------------------------
2019-11-26 01:09:40: xmlchange success <command> ./xmlchange CONTINUE_RUN=TRUE  </command>
 ---------------------------------------------------
2019-11-26 07:50:51: st_archive starting 
 ---------------------------------------------------
2019-11-26 07:51:00: st_archive success 
 ---------------------------------------------------
2019-11-26 08:03:23: case.run starting 
 ---------------------------------------------------
2019-11-26 08:03:27: model execution starting 
 ---------------------------------------------------
2019-11-26 11:56:55: model execution success 
 ---------------------------------------------------
2019-11-26 11:56:55: case.run success 
 ---------------------------------------------------
2019-11-26 11:57:01: xmlchange success <command> ./xmlchange CONTINUE_RUN=TRUE  </command>
 ---------------------------------------------------
2019-11-26 11:57:12: st_archive starting 
 ---------------------------------------------------
2019-11-26 11:57:23: st_archive success 
 ---------------------------------------------------
2019-11-26 15:53:02: case.run starting 
 ---------------------------------------------------
2019-11-26 15:53:06: model execution starting 
 ---------------------------------------------------
2019-11-26 19:46:41: model execution success 
 ---------------------------------------------------
2019-11-26 19:46:41: case.run success 
 ---------------------------------------------------
2019-11-26 19:46:46: xmlchange success <command> ./xmlchange CONTINUE_RUN=TRUE  </command>
 ---------------------------------------------------
2019-11-26 19:46:56: st_archive starting 
 ---------------------------------------------------
2019-11-26 19:47:05: st_archive success 
 ---------------------------------------------------
2019-11-26 19:51:21: case.run starting 
 ---------------------------------------------------
2019-11-26 19:51:25: model execution starting 
 ---------------------------------------------------
2019-11-26 23:44:20: model execution success 
 ---------------------------------------------------
2019-11-26 23:44:20: case.run success 
 ---------------------------------------------------
2019-11-26 23:44:25: xmlchange success <command> ./xmlchange CONTINUE_RUN=TRUE  </command>
 ---------------------------------------------------
2019-11-26 23:44:33: st_archive starting 
 ---------------------------------------------------
2019-11-26 23:44:43: st_archive success 
 ---------------------------------------------------
2019-11-27 00:06:12: case.run starting 
 ---------------------------------------------------
2019-11-27 00:06:16: model execution starting 
 ---------------------------------------------------
2019-11-27 03:49:34: model execution success 
 ---------------------------------------------------
2019-11-27 03:49:34: case.run success 
 ---------------------------------------------------
2019-11-27 03:49:39: xmlchange success <command> ./xmlchange CONTINUE_RUN=TRUE  </command>
 ---------------------------------------------------
2019-11-27 03:49:48: st_archive starting 
 ---------------------------------------------------
2019-11-27 03:49:56: st_archive success 
 ---------------------------------------------------
2019-11-27 03:50:15: case.run starting 
 ---------------------------------------------------
2019-11-27 03:50:19: model execution starting 
 ---------------------------------------------------
2019-11-27 07:27:05: model execution success 
 ---------------------------------------------------
2019-11-27 07:27:05: case.run success 
 ---------------------------------------------------
2019-11-27 07:27:10: xmlchange success <command> ./xmlchange CONTINUE_RUN=TRUE  </command>
 ---------------------------------------------------
2019-11-27 07:27:19: st_archive starting 
 ---------------------------------------------------
2019-11-27 07:27:27: st_archive success 
 ---------------------------------------------------
2019-11-27 07:27:40: case.run starting 
 ---------------------------------------------------
2019-11-27 07:27:44: model execution starting 
 ---------------------------------------------------
2019-11-27 11:06:24: model execution success 
 ---------------------------------------------------
2019-11-27 11:06:24: case.run success 
 ---------------------------------------------------
2019-11-27 11:06:29: xmlchange success <command> ./xmlchange CONTINUE_RUN=TRUE  </command>
 ---------------------------------------------------
2019-11-27 11:06:38: st_archive starting 
 ---------------------------------------------------
2019-11-27 11:06:48: st_archive success 
 ---------------------------------------------------
2019-11-27 11:31:19: case.run starting 
 ---------------------------------------------------
2019-11-27 11:31:22: model execution starting 
 ---------------------------------------------------
2019-11-27 15:09:15: model execution success 
 ---------------------------------------------------
2019-11-27 15:09:15: case.run success 
 ---------------------------------------------------
2019-11-27 15:09:20: xmlchange success <command> ./xmlchange CONTINUE_RUN=TRUE  </command>
 ---------------------------------------------------
2019-11-27 15:09:32: st_archive starting 
 ---------------------------------------------------
2019-11-27 15:09:42: st_archive success 
 ---------------------------------------------------
2019-11-27 19:57:19: case.run starting 
 ---------------------------------------------------
2019-11-27 19:57:23: model execution starting 
 ---------------------------------------------------
2019-11-27 23:35:51: model execution success 
 ---------------------------------------------------
2019-11-27 23:35:51: case.run success 
 ---------------------------------------------------
2019-11-27 23:35:56: xmlchange success <command> ./xmlchange CONTINUE_RUN=TRUE  </command>
 ---------------------------------------------------
2019-11-27 23:36:08: st_archive starting 
 ---------------------------------------------------
2019-11-27 23:36:17: st_archive success 
 ---------------------------------------------------
