===============================================================
Tag name:  ctsm1.0.dev108
Originator(s):  erik (Erik Kluzek,UCAR/TSS,303-497-1326)
Date: Wed Aug 19 17:23:47 MDT 2020
One-line Summary:  Update default PE layouts for new SE/FV3 grids

Purpose of changes
------------------

Change PE layouts for new high resolution SE/FV3 grids. Have a version for cheyenne and a default one
as well.

Update externals: cime, cism, CMEPS, CDEPS
Change single point tests to use stub ROF
Change tests to use gx1v7 rather than gx1v6
Update waccmx_offline to use START=1979

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): 
  Fixes #1108 Change single point tests to use stub ROF
  Fixes #1098 ARCTICGRIS PE layout is very slow
  Fixes #1105 Default new FV3/SE grid PE layouts are problematic
  Fixes #1103 Some changes to prevent missing file for WACCMX testing
  Fixes #1113 ARCTIC test is failing

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations? No
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): None

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): None

Changes made to namelist defaults (e.g., changed parameter values): None

Changes to the datasets (e.g., parameter, surface or initial files): None

Substantial timing or memory changes: None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance):
   Some tests were chagned to use stub ROF and one to stub GLC. @billsacks suggest we should also change
   other tests in this same way.

Changes to tests or testing: Yes

Code reviewed by: self


CTSM testing: regular, tools

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS

  tools-tests (test/tools):

    cheyenne - PASS

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - OK

  python testing (see instructions in python/README.md; document testing done):

   cheyenne -- PASS

  regular tests (aux_clm):

    cheyenne ---- PASS
    izumi ------- PASS

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: No (except for waccmx_offline test)

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): cime, cism, CDEPS, CMEPS
   cism to cism2_1_69
   cime to cime5.8.30
   Update hash for CDEPS and CMEPS (so that nuopc test will run)

Pull Requests that document the changes (include PR ids): #1111
(https://github.com/ESCOMP/ctsm/pull)

  #1111 -- Adjust FV3/SE PE layouts

===============================================================
===============================================================
Tag name:  ctsm1.0.dev107
Originator(s):  erik (Erik Kluzek)
Date: Mon Aug 10 02:21:12 MDT 2020
One-line Summary: Answer changes needed for CESM2.2.0

Purpose of changes
------------------

A list of small answer changes needed for CESM2.2.0. Nitrogen deposition file now properly
ends in 2015 rather than 2005. Have 2010_control compset match the 2000 IOC file rather than 2010.
Modify the waccmx_offline tests to more closely match the CAM tests.

Interpolate IC files for an exact match. Answers are identical for an exact match, but different
when used for other resolutions.

Two small bit-for-bit changes remove levurb and string_len dimensions on restart files.
Remove xsmrpool_loss when use_crop is NOT true.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): [If none, remove this line]
  Answer changing fixes:
  Fixes #1004 -- stream_year_last_ndep is set to 2005 instead of 2015 for historical with ctsm1.0.dev093 
  Fixes #1012 -- Help prevent problems with WACCM-X testing by modifying our waccmx_offline test
  Fixes #1090 -- Have 2010_control compset match 2000 simulation-year rather than 2010
  Fixes #1089 -- Failure in FXHIST test when running with CAM 
  Fixes #1037 -- Interpolate out-of-the-box initial conditions files for CESM2.2 release
                 (only changes answers for resolutions other than IC file resolution)

  Bit-for-bit changes:
  Fixes #1101 -- levurb and string_len dimensions no longer needed on the restart files
  Fixes #1097 -- xsmrpool_loss should only be written to the restart file if use_crop is true

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations? No
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): None

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables):
   2010_control doesn't match 2010 finidat file

Changes made to namelist defaults (e.g., changed parameter values): None

Changes to the datasets (e.g., parameter, surface or initial files): Interpolated IC files
   Add f19 2003 IC file, f09 and 19 1979 IC file
   Remove 2010 IC file

Substantial timing or memory changes: None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): None

Changes to tests or testing: waccm_offline test changed to more closely match the FXHIST CAM test

Code reviewed by: self


CTSM testing: regular

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS (115 tests are different than before)

  python testing (see instructions in python/README.md; document testing done):

    cheyenne - PASS

  regular tests (aux_clm):

    cheyenne ---- PASS
    izumi ------- PASS

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: Yes!

  Hist and SSP BGC compsets change answers because of the correction of the end year for ndep. 
  You wouldn't think this would matter until you get to the end of the time-sequence -- but it 
  does for the first time-step.

  The 2010 compset changes answers because of using the 2000 finidat file rather than 2010

  Some finidat files interpolated to the resolution they are designed for. This causes an apparent 
  change of answers when these datasets are used for other resolutions (even though it does NOT for 
  the resolution it's interpolated to)

  waccm_offline tests are also changed in order to more closely match the FXHIST tests in CAM.

  Summarize any changes to answers, i.e.,
    - what code configurations: Hist, SSP, 2010 compsets, Clm45 and some f10, 1x1 because of IC files
    - what platforms/compilers: All
    - nature of change: same climate

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): None

Pull Requests that document the changes (include PR ids): #1100
(https://github.com/ESCOMP/ctsm/pull)
   #1100 -- Answer changes needed for CESM2.2.0

===============================================================
===============================================================
Tag name:  ctsm1.0.dev106
Originator(s):  erik (Erik Kluzek)
Date: Thu Aug  6 15:35:47 MDT 2020
One-line Summary: Bit-for-bit updates for the CESM2.2.0 release

Purpose of changes
------------------

Fix some bit-for-bit things needed for the CESM2.2.0 release
Bring changes needed from release-clm5.0.31-34 to trunk. ndep change.
Brings in some changes for PtVg datasets for no-anthro compset.
Do some refactoring of Fire class so that it makes sense for FATES to use the base class.
Update RTM and MOSART with NUOPC changes and an update of a NetCDF-4 file to NetCDF-5
needed for the cesm2.2.0 release.

The 2010 finidat file is now set in the 2010_control use-case rather than relying on the
finidat dataset matching.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): 
 Fixes #1087 -- wrong finidat file
 Fixes #1096 -- failing tests
 Fixes #1036 -- ne30 case fails
 Fixes #946 --- better error message
 Fixes #983 --- Fire method renames
 Fixes #938 --- mscripgrid issue

Known bugs found since the previous tag (include github issue ID): 
  #1098 -- ARCTICGRIS PE-layout is very slow...
  #1097 -- xsmrpool_loss should only be written to the restart file if use_crop is true


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations? No
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions):

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables):
   if ndep file isn't found die with a warning rather than fatal error.
   The warning allows you to override with CLM_BLDNML_OPTS="-ignore_warnings"

   finidat file matching will now use the sim_year rather than start-year
   for a control case (transient cases used the start-year)
   Add 2010 f09 finidat file for clm5_0_cam6.0

Changes made to namelist defaults (e.g., changed parameter values):

Changes to the datasets (e.g., parameter, surface or initial files):

Substantial timing or memory changes: None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance):

Changes to tests or testing: 
   Add clm45cam6LndTuningMode testmod
   Some tests needed more wallclock

Code reviewed by: self


CTSM testing: regular

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS (one test is different than ctsm1.0.dev105)

  python testing (see instructions in python/README.md; document testing done):

    cheyenne -= PASS

  regular tests (aux_clm):

    cheyenne ---- PASS
    izumi ------- PASS

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: No (bit-for-bit)

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): rtm, mosart
  RTM    to rtm1_0_72
  MOSART to mosart1_0_37

Pull Requests that document the changes (include PR ids): #1079
(https://github.com/ESCOMP/ctsm/pull)
  #1079 -- Bit-for-bit updates for the CESM2.2.0 release

===============================================================
===============================================================
Tag name:  ctsm1.0.dev105
Originator(s):  erik (Erik Kluzek,UCAR/TSS,303-497-1326) / Chris Fischer
Date: Thu Jul 23 08:59:37 MDT 2020
One-line Summary: Bring in some new SE grids for CAM as well as initial condition files for them

Purpose of changes
------------------
   Add new SE (Spectral Element) grids: ne30np4.pg2, ne30np4.pg3, ne30np4, ne60np4, ne120np4, ne120np4.pg2, ne120np4.pg3, 
                                        ne0np4CONUS.ne30x8, ne0np4.ARCTIC.ne30x4, ne0np4.ARCTICGRIS.ne30x8
   Add new datasets for them, some new tests, and add support for their creation.

   Also add some new initial condition (IC) files for 1979, 2000, and 2013 for use when coupled to CAM

   The process for picking initial condition files was improved so that there can be more than one year that matches
   and it will do the best to pick the best match. Also added the possibility for 1982 and 2013 simulation years for
   specific grid matches.

  Also update cime to a newer version that supports these new grids. And update CMEPS (and hence add in fox, CDEPS)
  to latest used on the nuopc_dev branch. Testing with nuopc wouldn't work without this update. The cime version updates
  the tables for dry-deposition and changes answers for dry-deposition when it's turned on.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): #992 #994
   Fixes #992 -- New initial conditions for 1979, 2000
   Fixes #994 -- New CAM SE grids
   Fixes #888 -- Add support for various "physics grids" needed by CAM
CIME Issues fixed (include issue #): #3593, #3569, #3564, PR#3557
   cime/#3593 -- Fix mapping files for new grids
   cime/#3569 -- Fix cime regression testing
   cime/#3564 -- Change alias for arctic grids
   cime/#3557 -- updates to dry deposition data (PR)

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations? No
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions):
  Initial conditions (IC) for 1979 start date is technically 1982
  New initial condition files are ONLY for when coupled to CAM (B and F compsets)

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): Allow for new grids
   User updates to cime:
    - Allow a user supplied config_grids.xml file.
    - Improved error message for incorrect setting of cime model.

Changes made to namelist defaults (e.g., changed parameter values): Add IC files for 1982 and 2013 years

Changes to the datasets (e.g., parameter, surface or initial files): New SE grid datasets
   Remove conus_30_x8 grid (previous name for new ne0np4CONUS.ne30x8 grid
   Add new grids: ne30np4.pg2, ne30np4.pg3, ne30np4, ne60np4, ne120np4, ne120np4.pg2, ne120np4.pg3, 
                  ne0np4CONUS.ne30x8, ne0np4.ARCTIC.ne30x4, ne0np4.ARCTICGRIS.ne30x8
   Add new mask: tx0.1v3
   New IC files: 1982 for ARCTIC and ARCTICGRIS, ne120np4 for 2000 and 2013 for CONUS all when coupled to CAM

Substantial timing or memory changes: None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance):
   Initial condition picking will pick the file it finds with the closet simulation year to the request
   for the lnd_tuning_mode desired. It will pick based on year over an exact match in resolution.
   You can get around this by setting use_init_interp to be specific for a resolution

   Also realized that there is a dependence between the namelist_defaults_ctsm.xml file and namelist_defaults_overall.xml
   for the clm_start_type item. It must be coordinated with use_init_interp, init_interp_attributes and finidat
   to work properly. I added a comment about this. We should probably just move them into the same file.

Changes to tests or testing: Added tests for new grids

Code reviewed by: self, fischer-ncar, adamrher


CTSM testing: regular

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - OK (130 tests fail comparison to previous verison due to new IC files)

  tools-tests (test/tools):

    izumi - OK

  PTCLM testing (tools/shared/PTCLM/test):

    izumi - OK

  python testing (see instructions in python/README.md; document testing done):

   izumi -- PASS

  regular tests (aux_clm):

    cheyenne ---- OK
    izumi ------- PASS

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: No bit-for-bit (unless running with CAM because of new finidat files)
    Running with nuopc changes answers because of updates to nuopc drivers
    Dry-deposition changes answers when turned on (but doesn't effect anything else)

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): cime, cmeps, cdeps, fox
    cime updated to cime5.8.28 (from branch_tags/cime5.8.24_a01)
    fox, and cdeps added in
    cmeps, cdeps and fox updated to hashes used in the nuopc_dev branch of CESM

Pull Requests that document the changes (include PR ids): #1038
(https://github.com/ESCOMP/ctsm/pull)
   #1038 -- New fsufdat and landuse time series files for ne grids

===============================================================
===============================================================
Tag name:  ctsm1.0.dev104
Originator(s):  sacks (Bill Sacks)
Date:  Mon Jul  6 09:58:15 MDT 2020
One-line Summary: Add LILAC

Purpose of changes
------------------

Add LILAC: The Lightweight Infrastructure for Land-Atmosphere
Coupling. This infrastructure consists of two major pieces:

(1) A lightweight coupling infrastructure built on top of ESMF that
    makes it easier for atmosphere models to call CTSM directly, rather
    than using the hub-and-spoke architecture that is used by CESM.

(2) A set of python-based tools for building CTSM and creating its
    runtime inputs when running in an atmosphere model via
    LILAC. Although these tools are built on top of cime, details of the
    create_newcase / case.setup / case.build process are hidden from the
    user, because many of the aspects of this workflow don't make sense
    in the LILAC context.

So far we have used LILAC to couple CTSM to WRF. There are plans to use
the same infrastructure to couple CTSM to other regional atmosphere
models.

Documentation of LILAC is provided in
https://escomp.github.io/ctsm-docs/versions/master/html/lilac/index.html
(though there are still some missing sections), as well as in various
presentations on the wiki
(https://github.com/ESCOMP/CTSM/wiki/Presentations).

There have been many contributors besides myself to the development,
testing and documentation of LILAC; chief among them being Mariana
Vertenstein, Negin Sobhani, Joe Hamman, Sam Levis, Mike Barlage and Dave
Lawrence.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- See issues in the Done column of https://github.com/ESCOMP/CTSM/projects/23

Known bugs introduced in this tag (include github issue ID):
- Although LILAC is working to first order, there is still some work to
do. See outstanding issues in https://github.com/ESCOMP/CTSM/projects/23


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables):
- dtime no longer specified on the namelist: just obtained from driver

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing:
- Added LILACSMOKE test

Code reviewed by: self


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - ok

    Baseline comparisons fail as expected. One test failed, but it also
    failed for me on master (https://github.com/ESCOMP/CTSM/issues/1074)

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - pass on my mac

  regular tests (aux_clm):

    cheyenne ---- pass
    izumi ------- pass

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: NO


Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
https://github.com/ESCOMP/CTSM/pull/1068

===============================================================
===============================================================
Tag name:  ctsm1.0.dev103
Originator(s):  slevis (Samuel Levis, SLevis Consulting LLC,303-665-1310)
Date:  Mon Jun 29 17:16:29 MDT 2020
One-line Summary: Gridcell-level balance-check for methane (CH4)

Purpose of changes
------------------

 Bracket the model time-step loop to calculate balance checks at the
 gridcell level, as detailed in issue #315. The column-level check
 remains unchanged.

 Subroutine ch4_init_balance_check is replaced with
 ch4_init_column_balance_check. Subroutine
 ch4_init_gridcell_balance_check is added.

 The implementation is similar to the one for carbon and nitrogen (see
 tag ctsm1.0.dev096).

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): #315


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions):
 None

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables):
 None

Changes made to namelist defaults (e.g., changed parameter values):
 None

Changes to the datasets (e.g., parameter, surface or initial files):
 None

Substantial timing or memory changes:
 No

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance):
 None

Changes to tests or testing:
 None

Code reviewed by:
 @billsacks


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - 

  tools-tests (test/tools):

    cheyenne - 

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - 

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - 

  regular tests (aux_clm):

    cheyenne ---- PASS
    izumi ------- PASS

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline:
 No

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.):
 None

Pull Requests that document the changes (include PR ids):
 https://github.com/ESCOMP/CTSM/pull/1022

===============================================================
===============================================================
Tag name:  ctsm1.0.dev102
Originator(s):  erik (Erik Kluzek,UCAR/TSS,303-497-1326)
Date: Fri Jun 26 01:32:04 MDT 2020
One-line Summary: Some important fixes for LUNA in clm5_0, and small urban issue in clm5_0

Purpose of changes
------------------

Fix some important issues that were found in LUNA for the arctic. These changes were large enough
that we had to re-tune the clm5_0 parameters to give a similar simulation. Leah Birch noticed 
that the formulations in the code disagreed with the paper. Correcting this caused arctic plants
to be less productive. In order to compensate parameters for leafcn, and slatop were adjusted for 
broadleaf_deciduous_boreal_shrub and c3_arctic_grass for clm5_0.

The total vegetation carbon threshold on exit_spinup (from accelerated decomposition (AD) mode) was 
increased from 0.1 to 1.0. Without this change some PFT's can die out in exit spinup even when
they have significant carbon stores from the AD spinup mode.

There is an implicit assumption in the urban building energy model that building width equals 
street width. However, this assumption can/should be relaxed and building width can be derived 
from the morphology dataset. 


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):  #738 #803 #953 #958 #1056
 Fixes #1056 -- Remove allpfts dimension
 Fixes #953 -- Incorrect formula in LUNA uses day time rather than average of day/night
 Fixes #958 -- LUNA day length factor missing
 Fixes #738 -- Totvegc threshold increased from 0.1 to 1.
 Fixes #803 -- No longer assume building width

Known bugs introduced in this tag (include github issue ID): #1060
   #1060 -- Some code cleanup of luna bug fixes

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[x] clm5_0

[x] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): clm5_0 climate adjusted for arctic plants

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values):

Changes to the datasets (e.g., parameter, surface or initial files): New parameter files

Substantial timing or memory changes: None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): We 
  chose some bad names for variables, so we'll clean this up in a seperate step (see #1060)

Changes to tests or testing: None

Code reviewed by: self, olyson, bill-sacks, wweider


CTSM testing: regular

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS (266 compare differently because of update in paramsfile and jmaxb1)

  python testing (see instructions in python/README.md; document testing done):

    cheyenne - PASS

  regular tests (aux_clm):

    cheyenne ---- PASS
    izumi ------- PASS

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline:  Yes for clm5_0

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: clm5_0 when Luna on as well as smaller changes for urban
    - what platforms/compilers: all
    - nature of change: climate changing, but tuned to respond similar to clm5.0

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
     luna changes: oleson case clm50_ctsm10d089_2deg_GSWP3V1_luna3_jmaxb1-0.17_slatopA_leafcnA_hist

   URL for LMWG diagnostics output used to validate new climate:

    urban changes:


http://webext.cgd.ucar.edu/I2000/clm50_ctsm10d098_1deg_GSWP3V1_CON_FIXBUILDENERGY_2000/lnd/clm50_ctsm10d098_1deg_GSWP3V1_CON_FIXBUILDENERGY_2000.1991_2010-clm50_ctsm10d098_1deg_GSWP3V1_CON_2000.1991_2010/setsIndex.html
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): None

Pull Requests that document the changes (include PR ids): #1034 and #962
(https://github.com/ESCOMP/ctsm/pull)
   #1034 -- Building energy fix
   #962 --- Fixes for the LUNA dayl bugs 

===============================================================
===============================================================
Tag name:  ctsm1.0.dev101
Originator(s):  erik (Erik Kluzek,UCAR/TSS,303-497-1326), oleson (Keith Oleson)
Date: Wed Jun 17 23:51:22 MDT 2020
One-line Summary: Changes from Keith to bring a list of variables to the parameter file

Purpose of changes
------------------

Change some hardcoded parameters to go on the parameter files. This is needed in preparation 
of running the Perturbed Parameter Ensemble.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): #995 #138, #206

Fixes #995 -- Soil turnover times are on paramfile and hardcoded
Fixes #138 -- use tau_ values on paramfile
Fixes #206 -- add prh30 to paramfile
Some work on #290 -- LUNA refactoring


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations? No
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): None

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): None

Changes made to namelist defaults (e.g., changed parameter values): None

Changes to the datasets (e.g., parameter, surface or initial files): paramfile for both clm4_5 and clm5_0
   Add 51 new parameters to the paramsfile:
       bsw_adjustfactor,cpha,csol_clay,csol_om,csol_sand,denitrif_nitrateconc_coefficient,denitrif_nitrateconc_exponent,
       denitrif_respiration_coefficient,denitrif_respiration_exponent,drift_gs,e_ice,enzyme_turnover_daily,eta0_anderson,
       eta0_vionnet,fnps,hksat_adjustfactor,ignition_efficiency,jmaxb0,jmaxha,jmaxhd,kcha,koha,kp25ratio,lmrha,lmrhd,
       lmrse,luna_theta_cj,minrelh,n_baseflow,pd,perched_baseflow_scalar,prh30,relhExp,ssi,sucsat_adjustfactor,theta_cj,
       theta_ip,theta_psii,tpuha,tpuhd,tkd_clay,tkd_om,tkd_sand,tkm_om,tpu25ratio,vcmaxha,vcmaxhd,watsat_adjustfactor,
       wc2wjb0,wimp,wind_min
    k_nitr_max removed and replaced with k_nitr_max_perday
    meta-data for the tau_* variables changed, and a roundoff difference in the clm4_5 paramfile

Substantial timing or memory changes: None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): prh30 not used for clm4_5
   prh30 was added in but won't be used for the CNFireLi2014 formulation. 
   It's only used for the clm5_0 and following formulations

Changes to tests or testing: None

Code reviewed by: self, oleson


CTSM testing: regular

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS (170 tests differ from baseline because of new paramfiles)

  python testing (see instructions in python/README.md; document testing done):

    cheyenne - PASS

  regular tests (aux_clm):

    cheyenne ---- OK
    izumi ------- OK

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: No bit-for-bit

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): None

Pull Requests that document the changes (include PR ids): #1039
(https://github.com/ESCOMP/ctsm/pull)
    #1039 -- move hardcoded parameters to paramfile

===============================================================
===============================================================
Tag name:  ctsm1.0.dev100
Originator(s):  sacks (Bill Sacks)
Date:  Tue Jun  9 06:15:15 MDT 2020
One-line Summary: Deallocate memory after running init_interp
 
Purpose of changes
------------------

Many of the pointer arrays in init_interp were not being deallocated
when init_interp finished. This was a particular problem because some of
these arrays were allocated for the full global source grid on each
processor, so the memory use was significant. This tag fixes this issue.

This issue is documented in https://github.com/ESCOMP/CTSM/issues/1010
and https://github.com/ESCOMP/CTSM/issues/1032.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/CTSM#1032 (Memory isn't deallocated in some of the
  init_interp code)

Known bugs found since the previous tag (include github issue ID):
- This does NOT resolve ESCOMP/CTSM#1010 (WACCMX test fails with
  ctsm1.0.dev093 in cesm2_2_alpha05c), even though that was the original
  hope.


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none
- The initial run of the PFS test showed a very long time taken in
  initialization. However, it appears that this was a system issue:
  rerunning twice showed times back to normal.

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: self


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - pass (on my mac)

  regular tests (aux_clm):

    cheyenne ---- pass
    izumi ------- pass

    Most testing was done on f332fafc. After that, I made a minor change
    that was needed for the unit tests to pass. I then reran (1)
    clm_short on cheyenne, and (2) this subset of the test list on
    izumi:

      SMS.f10_f10_musgs.I2000Clm50BgcCrop.izumi_gnu.clm-crop
      SMS_D.f10_f10_musgs.I2000Clm50BgcCrop.izumi_gnu.clm-crop
      PEM_Ld1.f10_f10_musgs.I2000Clm50BgcCropGs.izumi_intel.clm-crop
      SMS_D.f10_f10_musgs.I2000Clm50BgcCrop.izumi_intel.clm-crop
      SMS.f10_f10_musgs.I2000Clm50BgcCrop.izumi_pgi.clm-crop
      SMS_D.f10_f10_musgs.I2000Clm50BgcCrop.izumi_pgi.clm-crop
      SMS_Ld5_D_P48x1.f10_f10_musgs.IHistClm50Bgc.izumi_nag.clm-monthly
      ERP_Ld5_P48x1.f10_f10_musgs.I1850Clm50Bgc.izumi_nag.clm-default

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: NO


Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev099
Originator(s):  sacks (Bill Sacks)
Date:  Mon Jun  8 09:29:38 MDT 2020
One-line Summary: Update cime, needed for izumi machine updates

Purpose of changes
------------------

Update cime. The most important change here is an update for izumi,
which is needed for the recent izumi machine updates.

Also: Change izumi PEM test to use SGLC (after the machine updates, the
PEM test was failing; see also
https://github.com/ESCOMP/CTSM/issues/840).

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: not investigated; none expected

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing:
- Changed izumi PEM test to use SGLC (after the machine updates, the PEM
test was failing; see also https://github.com/ESCOMP/CTSM/issues/840).

Code reviewed by: self


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- pass
    izumi ------- ok

    izumi: tests pass, some baselines fail as expected

    ERP_D_Ld3.f09_g17.I2000Clm50SpGs.cheyenne_intel.clm-prescribed
    showed differences from the stored baselines, but it appears that
    the stored baselines were incorrect: when I reran this test from
    master, the master version differed from the stored baselines and
    was identical to this branch.

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES, just on izumi due to machine updates

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: some izumi tests
    - what platforms/compilers: izumi gnu & pgi
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      not investigated; expected to be roundoff

      Answer changes were observed in these tests:
      - SMS_D.f10_f10_musgs.I2000Clm50BgcCrop.izumi_gnu.clm-crop
      - SMS_D.f10_f10_musgs.I2000Clm50BgcCrop.izumi_pgi.clm-crop
      - SMS.f10_f10_musgs.I2000Clm50BgcCrop.izumi_pgi.clm-crop

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.):
- cime: branch_tags/cime5.8.23_a01 -> branch_tags/cime5.8.24_a01
    Point to a cime branch tag that is equivalent to cime5.8.24 but uses
    pio1 rather than pio2, even for mpi-serial cases (the changes in
    branch_tags/cime5.8.24_a01 relative to cime5.8.24 are equivalent to the
    changes in branch_tags/cime5.8.23_a01 relative to cime5.8.23).

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev098
Originator(s):  John Truesdale,UCAR/AMP,303-497-1386
Date: Thu May 28 14:39:27 MDT 2020
One-line Summary: Bring in GFDL fv3 cubed sphere grids C24, C48, C96, C192, C384

Purpose of changes
------------------

Add support for new GFDL fv3 curbed sphere grids. Low resolution C24, and C48
Moderate resolution C96, and high resolution C192 and C384. Add in mapping files
so all can be created. Make C96 a default resolution created with new surface
datasets. For all of these only the crop versions of surface and landuse.timeseries
are in place, as non-crop can run in CLM without needing different datasets. Also for
historical landuse.timeseries we use the SSP5-8.5 timeseries files, so that it can be
used for future, historical and beyond 2015 present day.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): #1025
 Fixes #1025 -- By default only create crop version of surface/landuse.timeseries datasets

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): None

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): None

Changes made to namelist defaults (e.g., changed parameter values): None

Changes to the datasets (e.g., parameter, surface or initial files): New surface and landuse.timeseries files

Substantial timing or memory changes: None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): None

Changes to tests or testing: Add C96 tests

Code reviewed by: @ekluzek

CTSM testing: regular, tools

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS (15 new tests for new resolutions)

  tools-tests (test/tools):

    cheyenne - OK

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - PASS (cheyenne)

  regular tests (aux_clm):

    cheyenne ---- PASS
    izumi ------- PASS

If the tag used for baseline comparisons was NOT the previous tag, note that here: previous


Answer changes
--------------

Changes answers relative to baseline: No bit-for-bit

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): None

Pull Requests that document the changes (include PR ids): #891
(https://github.com/ESCOMP/ctsm/pull)

   #891 -- add definitions/defaults for GFDL fv3 cubed sphere grids C24, C48, C96, C192, C384

===============================================================
===============================================================
Tag name:  ctsm1.0.dev097
Originator(s): ycheng (Yanyan Cheng); final changes & testing by Bill Sacks
Date:  Mon May 25 15:14:56 MDT 2020
One-line Summary: Add two bioenergy crops (switchgrass and miscanthus)

Purpose of changes
------------------

Main change is from Yanyan Cheng: adding parameters and code for two
bioenergy crops, switchgrass and miscanthus. Along with this, there is a
new potential flux from crop leaves and stems to the crop product pool
at harvest, representing biofuel products; a new pft-specific parameter
controls this flux (biofuel_harvfrac). Currently, the out-of-the-box
surface datasets do not specify any area for these crops, but the new
parameter file will allow them to be present if specified on the surface
dataset or landuse_timeseries file. Note that this is only an option for
CLM5.0, NOT for CLM4.5. (See https://github.com/ESCOMP/CTSM/pull/884 for
details.)

Also, some minor fixes from Bill Sacks: See notes under "bugs fixed"
below.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/CTSM#203 - Fixes creation of harvest-related variables
  on surface datasets created with the all_veg option - smallville,
  PTCLM, etc. (documented in https://github.com/ESCOMP/CTSM/pull/1019)
- Resolves ESCOMP/CTSM#930 - Makes gindex_ocn intent(inout) rather than
  intent(out)
- Resolves ESCOMP/CTSM#1021 - Changes SSP test to only do symlink if
  needed

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions):
- For transient crop cases, now more crops in memory. This creates a
  need for interpolation of initial conditions for initial conditions
  generated from a transient case.

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values):
- Changed parameter values for switchgrass and miscanthus (only for clm5.0)
- New pft-specific parameter, biofuel_harvfrac (for both clm4.5 and clm5.0)

Changes to the datasets (e.g., parameter, surface or initial files):
- New parameter files with the above changes

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing:
- Changed smallville ERS_Lm20 test to be cold start (see comment in
  commit 8368f222 for details)
- Added a multi-year smallville test that includes these new crops

Code reviewed by: Bill Sacks, Danica Lombardozzi


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - ok

    Pass except for expected baseline failures

    Ran on 633be0eb, with comparison against ctsm1.0.dev095; tests pass
    except these expected baseline failures:

        030 blf84 TBLscript_tools.sh PTCLM PTCLMmkdata PTCLM_USUMB_clm4_5^buildtools ....................\c
         rc=7 FAIL

        032 blfc4 TBLscript_tools.sh PTCLM PTCLMmkdata PTCLM_USUMB_Cycle_clm4_5^buildtools ..............\c
         rc=7 FAIL

        Diffs for these are just in the harvest-related fields on the surface dataset, as expected.

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    izumi ------- pass

    For cheyenne testing: a few tests had baseline failures. The
    differences just arise in the h1 (vector) files, due to differences
    in dimension sizes. These differences are expected. See below for
    details.

    Ran most testing on 8309c213. Reran two SSP tests on 4665c9ce

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES, but diagnostic only due to
change in dimension sizes - does not change answers in any meaningful
way

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: Transient CLM50 cases with crops; just
      changes dimensions on vector (h1) history files
    - what platforms/compilers: All
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      diagnostic change only

      Differences were observed in the following tests:

          ERS_Ly3_Mmpi-serial.1x1_smallvilleIA.IHistClm50BgcCropQianGs.cheyenne_gnu.clm-cropMonthOutput
          SMS_Ld5.f10_f10_musgs.ISSP245Clm50BgcCrop.cheyenne_gnu.clm-ciso_dec2050Start
          SMS_Ld5.f10_f10_musgs.ISSP370Clm50BgcCrop.cheyenne_gnu.clm-ciso_dec2050Start
          ERP_D_Ld10_P36x2.f10_f10_musgs.IHistClm50BgcCrop.cheyenne_intel.clm-ciso_decStart
          ERP_D_Ld5.f10_f10_musgs.IHistClm50BgcCrop.cheyenne_intel.clm-allActive
          ERP_Ly3_P72x2.f10_f10_musgs.IHistClm50BgcCrop.cheyenne_intel.clm-cropMonthOutput
          ERS_D_Ld7_Mmpi-serial.1x1_smallvilleIA.IHistClm50BgcCropGs.cheyenne_intel.clm-decStart1851_noinitial
          ERS_Ly3_P72x2.f10_f10_musgs.IHistClm50BgcCropG.cheyenne_intel.clm-cropMonthOutput
          ERS_Ly5_P144x1.f10_f10_musgs.IHistClm50BgcCrop.cheyenne_intel.clm-cropMonthOutput
          ERS_Ly6_Mmpi-serial.1x1_smallvilleIA.IHistClm50BgcCropQianGs.cheyenne_intel.clm-cropMonthOutput
          LCISO_Lm13.f10_f10_musgs.IHistClm50BgcCrop.cheyenne_intel.clm-ciso_monthly
          SMS_Ld5.f10_f10_musgs.ISSP585Clm50BgcCrop.cheyenne_intel.clm-ciso_dec2050Start

      The differences just arise in the h1 files, due to differences in
      dimension sizes. I reran these tests with a paramfile where I
      reverted the mergeToPft variable; then they were all bfb with
      master.

      Other than this, there were FIELDLIST differences, as expected.

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
- https://github.com/ESCOMP/CTSM/pull/884
- https://github.com/ESCOMP/CTSM/pull/1019

===============================================================
===============================================================
Tag name:  ctsm1.0.dev096
Originator(s):  slevis (Samuel Levis,SLevis Consulting LLC,303-665-1310)
Date:  Thu May 21 19:18:24 MDT 2020
One-line Summary: Gridcell-level balance checks for carbon and nitrogen

Purpose of changes
------------------

 Bracket the model's time-step loop to calculate balance checks at the
 gridcell level because there are terms in the carbon and nitrogen
 cycles not accounted for at the column level. Balance checks at the
 column level remain unchanged for now.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): #314


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): None

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): None

Changes made to namelist defaults (e.g., changed parameter values): None

Changes to the datasets (e.g., parameter, surface or initial files): None

Substantial timing or memory changes: None 

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): None

Changes to tests or testing: None

Code reviewed by:
 @billsacks

CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - 

  tools-tests (test/tools):

    cheyenne - 

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - 

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - 

  regular tests (aux_clm):

    cheyenne ---- OK
    izumi ------- PASS

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: No


Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): None

Pull Requests that document the changes (include PR ids):
 https://github.com/ESCOMP/CTSM/pull/984

===============================================================
===============================================================
Tag name:  ctsm1.0.dev095
Originator(s):  sacks (Bill Sacks)
Date:  Thu May 21 10:25:16 MDT 2020
One-line Summary: Update cime and cmeps externals; rework initialization of CNFire object

Purpose of changes
------------------

Main change is to update cime and cmeps externals. cime is now
essentially at the version used in cesm2_2_beta05 (but with one change
backed out); cmeps is at latest master.

Also, reworks initialization of CNFire object to avoid occasional
segmentation faults.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/CTSM#1018 (Occasional seg faults when initializing
  CNFire object)

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none
- Note: Because of a change in cime, datm now uses netcdf rather than
  pnetcdf, but it doesn't seem that that impacts performance
  significantly

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: self


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    izumi ------- pass

    cheyenne tests pass; some answer changes as expected (as noted below)

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations:
      - cheyenne_gnu - due to compiler version update
      - nuopc - presumably due to the major cmeps update
    - what platforms/compilers: see above
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      - Not investigated

    I have verified that gnu tests in 59d1dd4a are identical to those
    in dcaad375 - i.e., the rework of the initialization logic for fire
    didn't change answers

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.):
- cime: cime5.8.19 -> branch_tags/cime5.8.23_a01
  (same as cime5.8.23, but reverts to using pio1 for mpi-serial cases,
  since some mpi-serial cases failed with pio2)
- cmeps

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev094
Originator(s):  sacks (Bill Sacks)
Date:  Fri May 15 13:41:01 MDT 2020
One-line Summary: Minor bug fixes needed for latest cime

Purpose of changes
------------------

(1) Refactor a conditional to avoid divide by 0 (ESCOMP/CTSM#1013)

(2) Remove some FATES coordinate variables that were writing garbage
    (temporary workaround for ESCOMP/CTSM#1014)

(3) Minor update to manage_externals

(4) Document authorship in README.md and .zenodo.json


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/CTSM#1013
- Workaround for ESCOMP/CTSM#1014, but doesn't properly resolve that issue

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: not checked; none expected

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: self


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- pass
    izumi ------- pass

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: NO

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
- https://github.com/ESCOMP/ctsm/pull/1006

===============================================================
===============================================================
Tag name:  ctsm1.0.dev093
Originator(s):  erik (Erik Kluzek)
Date: Fri May  1 22:27:16 MDT 2020
One-line Summary: Bring in changes from release-clm5.0 branch to master

Purpose of changes
------------------

Move changes from release-clm5.0.12.13/14/16/17/18/20/21/22/23/24/25/26/27/28/29/30 to master
(note excludes release-clm5.0.15, release-clm5.0.19)

Answer changing!
Updates FATES with PARTEH option.

Remove CRUNCEP half degree, ne120np4, f05 surface datasets
Landuse-timeseries files for just: f09, f19, f10, ne30np4, 1x1_numaIA, 1x1_smallvilleIA, 1x1_brazil
Add datasets needed for SSP scenarios (only Tier I for ndep). Add ZWT_inversion dataset for methane.

Add new ability to use prescribed soil moisture data (from streams files based on an earlier CTSM simulation).

Add new MODISv2 mapping files for mksurfdata_map. Remove T5 and T21 mapping files. Add conus_30_x8 mapping files,
and T62 mapping files

Add contrib directory to tools with contributed tools and scripts.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): 31 issues....
  #873 -- Release clm5.0.12 changes
  #871 -- don't check for valid resolution for CESM cases by default
  #836 -- add izumi as a place that can run tools tests
  #831 -- add namelist tint_algo settings for input streams
  #817 -- Have clm4_5 use the same ndep file as clm5_0 for 1850
  #809 -- mkmapdata using an old version of intel
  #804 -- mksurfdata option
  #791 -- fix threading for prescribed LAI and soil-moisture
  #786 -- allow use_init_interp for a branch
  #746 -- Some CLM45 configurations use CLM50 initial conditions files
  #741 -- C13_HR, C13_NBP, FPI values result in numeric conversion not representable error (changes answers)
  #734 -- Isotope historical files off of NetCDF-4
  #716 -- 2 degree CMIP6WACCMDECK needs Carbon isotopes off
  #707 -- Missing mapping files for hirespft
  #673 -- Ice shelf wetland fix in mksurfdata_map can lead to glacier+lake > 100% on surface datasets (changes surface dataset creation)
  #653 -- To save disk on fsurdat files use zwt0 on finindated file
  #629 -- Remove unused fields from surface datasets
  #624 -- QSNOEVAP history output incorrect
  #621 -- Fully remove rcp in place of only ssp_rcp
  #608 -- Odd (blocky) spatial patterns in generic crop leafc for BGC simulation when initialized from BGC-Crop (changes answers)
  #589 -- Existence of content in the lai_streams namelist makes it confusing to users
  #557 -- no-anthro compset and options
  #553 -- More robust mksurfdata_map logic for determining where to put wetlands
  #547 -- Add conus_30_x8 grid as valid option for CTSM and mksurfdata_map
  #545 -- Antarctica ice shelves are being treated as wetlands rather than glaciers
  #544 -- Transient cases are matching wrong IC file
  #533 -- Add -no-anthro option to mksurfdata
  #276 -- Tests for pre_alpha/pre_beta
  #262 -- hirespft option to mksurfdata.pl
  #175 -- Set stream_year_first_urbantv/last for CLM4.5
  #150 -- mksurfdata code cleanup

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): New surface datasets do mean initial conditions
  will need to be interpolated

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): Add 2010 compset option
   New namelist items: ndep_tintalgo, lai_tintalgo, lightng_tintalgo, urbantv_tintalgo, dribble_crophrv_xsmrpool_2atm
   New soil moisture namelist items: use_soil_moisture_streams, stream_year_first_soilm, stream_year_last_soilm, model_year_align_soilm,
                                     model_year_align_soilm, soilm_tintalgo, soilm_offset, soilm_ignore_data_if_missing
   New namelist items for FATES: fates_parteh_mode
   New 2010, no-anthro, SSP use cases
   New compsets: I2010Clm50Sp, I2000Clm50BgcCropQianGs, I1850Clm50BgcNoAnthro, I1850Clm50SpNoAnthro, IHistClm50SpCru, IHistClm50SpCru,
                 IHistClm45SpGs, IHistClm45SpGs
   New SSP compsets: ISSP585Clm50BgcCrop, ISSP126Clm50BgcCrop, ISSP126Clm50BgcCrop, ISSP245Clm50BgcCrop, ISSP370Clm50BgcCrop, 
                     ISSP434Clm50BgcCrop, ISSP460Clm50BgcCrop, ISSP460Clm50BgcCrop
   LAI streams options don't show up in the namelist unless you ask for them
   finundation comes from streams files, no longer from the surface dataset
   FATES adds a list of new history variables and dimension types

Changes made to namelist defaults (e.g., changed parameter values): New fsurdat files, SSP C13/C14 files
  Add PtVg surface dataset
  most finidat files now need to be interpolated, a few added in to get testing to work
  New ndep datasets

Changes to the datasets (e.g., parameter, surface or initial files):
  New fsurdat files, finidat files need to be interpolated from now

Substantial timing or memory changes: None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance):
   Threshold to reseed dead vegetation increased from zero to 1.0 as a hard coded parameter.
   Reseeding also happens for generic crop types and not just natural vegetation now.
   The ability to read in 3D streams was added (input files have to be at the same resolution as the model)

Changes to tests or testing:
   Add no-anthro, 2010, f19 waccm and SSP tests
   Make sure prealpha/prebeta tests are also run by aux_clm
   Add prescribed soil moisture test
   Lengthen waccm_offline test
   Remove hcru_hcru and edison tests
   Add some aux_cime_baseline tests

Code reviewed by: self


CTSM testing: regular, tools

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS (345 are different than baseline)

  tools-tests (test/tools):

    cheyenne - OK

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - OK

  python testing (see instructions in python/README.md; document testing done):

    cheyennne -- PASS (when ncar_pylib run)

  regular tests (aux_clm):

    cheyenne ---- OK
    izumi ------- OK

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: Yes

  Updates in all fsurdat files leads to changes in answers
  Also there are updated finidat files as they had to be interpolated
   to work with the new fsurdat files
  Reseeding change can effect answers on startup if activated
  Extra call to SoilBiogeochemPrecisionControl can cause a small answer change

  Summarize any changes to answers, i.e.,
    - what code configurations: all global
    - what platforms/compilers: all
    - nature of change: Main change is around Antarctica

  CMIP6 simulations and release branch simulations were run with these surface datasets

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): fates, PTCLM

    fates to sci.1.30.0_api.8.0.0
    PTCLM to PTCLM2_20200121

   FATES now points to the NGEET repository rather than the NCAR fates-release

Pull Requests that document the changes (include PR ids): #998, #883, #849, #567, #564, #561
(https://github.com/ESCOMP/ctsm/pull)
   #998 -- Remove Copyright file
   #883 -- Bring changes on release-clm5.0 branch to master PR status: ready priority: high type: enhancement
   #849 -- Fix link to new forums
   #567 -- Ignore patterns for vim
   #564 -- Update mksurfdata_map to include glaciers outside of pft landmask
   #561 -- Add list of checks that should be done when making new surface datasets

===============================================================
===============================================================
Tag name:  ctsm1.0.dev092
Originator(s):  erik (Erik Kluzek,UCAR/TSS,303-497-1326)
Date: Mon Apr 27 14:51:21 MDT 2020
One-line Summary: Update cime to version in cesm2_2_beta04

Purpose of changes
------------------

Update cime to latest in cesm2.2 beta tag (cesm2_2_beta04)
Fix two small issues that had gone on the release-clm5.0 branch.
Update manage_externals (to manic-v1.1.8)

NOTE: Part of the reason for updating cime, was because of some problems
Mike Mills was having on cheyenne with inputdata files being deleted out 
from under him before setting up his cases. We think system issues were 
interacting with issues with cime putting in place zero length
missing files for the version of cime he was using. Those issues went
away with a cime update, so we hope updating to a newer cime version here
will help prevent those issues for CTSM users.

cime updates:

some cleanup from the last maint-5.6 update
Add ne5np4.pg4 grid.
case.setup: Add --keep option
Add omip as option to DATM_CO2_TSERIES

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): #974 #966
   Fixes #974 -- python 3 problem in ssp test
   Fixes #966 -- a few history fields need to be made CF compliant
CIME Issues fixed (include issue #): 3459 3466
   3459 -- case.setup refuses to fix corrupted env_mach_specfic.xml
   3466 -- fix for aquap

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): None

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables):
   ./case.setup now has a --keep option

Changes made to namelist defaults (e.g., changed parameter values): None

Changes to the datasets (e.g., parameter, surface or initial files): None

Substantial timing or memory changes: None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): None

Changes to tests or testing: None

Code reviewed by: self


CTSM testing: regular

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS

  tools-tests (test/tools):

    cheyenne - OK

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - OK

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - 

  regular tests (aux_clm):

    cheyenne ---- OK
    izumi ------- OK

If the tag used for baseline comparisons was NOT the previous tag, note that here: previous


Answer changes
--------------

Changes answers relative to baseline: None bit-for-bit

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): cime
    cime to cime5.8.19
    (NOTE: CMEPS is pointing to a newer hash than used in the beta tag)

Pull Requests that document the changes (include PR ids):
(https://github.com/ESCOMP/ctsm/pull)
   #996 -- Update cime to latest cesm2.2 beta tag

===============================================================
===============================================================
Tag name:  ctsm1.0.dev091
Originator(s):  sacks (Bill Sacks)
Date:  Fri Apr 24 07:43:46 MDT 2020
One-line Summary: Increase tolerance on near-zero truncation for a snow state update

Purpose of changes
------------------

In UpdateState_TopLayerFluxes, the tolerance of 1.e-13 was occasionally
exceeded. Although I haven't done a careful analysis, it seems okay to
me to increase this tolerance slightly. Here I increase it to 1.e-12.

Previously, if top-layer h2osoi_ice or h2osoi_liq were reduced to less
than 1e-13 times the original value (in an absolute value sense), these
masses were set to 0; now we set these masses to 0 if they are reduced
to less than 1e-12 times the original. So we can now occasionally set a
value to exactly 0 when before it was left at slightly different from
zero. If the previous code led to a small positive value, between 1e-13
and 1e-12 times the original, this tag will change answers slightly. If
the previous code led to a small negative number, it would cause the
model to abort, leading to the issue reported in ESCOMP/CTSM#988; this
change should fix those occasional aborts.

This tag also introduces the general ability to set tolerances to a
custom value in calls to truncate_small_values.

See ESCOMP/CTSM#988 for details.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/CTSM#988


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none expected (not checked)

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: Erik Kluzek


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- pass
    izumi ------- pass

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES, in theory (but none in test suite)

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: potentially all, but very rarely
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      Roundoff

      Potentially changes answers by roundoff (see notes above), but no
      changes were observed in any test in the test suite

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
https://github.com/ESCOMP/CTSM/pull/989

===============================================================
===============================================================
Tag name:  ctsm1.0.dev090
Originator(s):  negins (Negin Sobhani,UCAR/CSEG,303-497-1224)
Date: Thu Apr 23 09:21:18 MDT 2020
One-line Summary: Refactoring banded diagonal matrix and solve system in SoilTemperature module

Purpose of changes
------------------

Previously, the SoilTemperature module has a high level of nested subroutine
calls, which hurt its performance and readability. There were many
redundancies, logical issues, and nested conditionals  in the modular code
as discussed in issue #323. In this PR, SoilTemperature module is optimized
and rewritten for better readability and performance while maintaining bfb
compatibility.


This PR addresses issue ESCOMP/ctsm#323


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
	-ESCOMP/ctsm#323

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: The revised code improved the
performance of the previous code by ~1.5-2x.

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: Bill Sacks


CTSM testing:


 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run 

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run 

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run 

  regular tests (aux_clm):

    cheyenne ---- OK 
    izumi ------- PASS

Externals used for the testing:
   [cism]
	  tag = cism2_1_68
   [rtm]
	  tag = rtm1_0_70
   [mosart]
	  tag = mosart1_0_35
   [cime]
	  tag = branch_tags/cime5.8.15_a01

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: NO

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
(https://github.com/ESCOMP/CTSM/pull/979)

===============================================================
===============================================================
Tag name:  ctsm1.0.dev089
Originator(s):  sacks (Bill Sacks)
Date:  Tue Apr  7 12:01:59 MDT 2020
One-line Summary: Bring documentation source to master

Purpose of changes
------------------

1. Bring documentation source to master: Pulls in the source from
   https://github.com/escomp/ctsm-docs. This is important so that
   documentation can remain in sync with changes in the model
   code. Images are stored here using git-lfs (Git Large File
   Storage). I also made some minor fixes to get the pdf build of the
   tech note working.

2. Use a different documentation theme that supports a version dropdown
   menu, and add the code needed to support this versioning on the
   documentation web pages. At a high level, the way the versioned
   documentation works is to have separate subdirectories in the
   gh-pages branch of the ctsm-docs repository for each version of the
   documentation we want to support. There is then a bit of JavaScript
   code which uses a json file in the gh-pages branch to determine which
   versions exist and how these should be named in the dropdown
   menu. Most of these changes were borrowed from ESMCI/cime#3439, which
   in turn borrowed from ESCOMP/CISM-wrapper#23, which in turn was a
   slight modification of an implementation provided by @mnlevy1981 for
   the MARBL documentation, which in turn borrowed from an
   implementation put together by Unidata (credit where credit is due).

   I am not aware of out-of-the-box support for a version pull-down in
   out-of-the-box sphinx themes (though the last time I looked was in
   Fall, 2018, so there may be something available now). However,
   support for a version dropdown exists in an open PR in the sphinx
   readthedocs theme repository: readthedocs/sphinx_rtd_theme#438. I
   have pushed this branch to a new repository in ESMCI
   (https://github.com/ESMCI/sphinx_rtd_theme) to facilitate long-term
   maintenance of this branch in case it disappears from the official
   sphinx_rtd_theme repository. I have also cherry-picked a commit onto
   that branch, which is needed to fix search functionality in sphinx1.8
   (from readthedocs/sphinx_rtd_theme#672) (which is another reason for
   maintaining our own copy of this branch). The branch in this
   repository is now named version-dropdown-with-fixes (branching off of
   the version-dropdown branch in the sphinx_rtd_theme repository). In
   the long-term, I am a little concerned about using this theme that
   isn't showing any signs of being merged to the main branch of the
   readthedocs theme, but this has been working for us in other projects
   for the last 2 years, so I feel this is a reasonable approach in the
   short-medium term.

The new process for building the documentation is given here:
https://github.com/ESCOMP/CTSM/wiki/Directions-for-editing-CLM-documentation-on-github-and-sphinx

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#239 (Move documentation source to master)

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: Erik Kluzek


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- not run!
    izumi ------- not run!

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: NO

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
https://github.com/ESCOMP/CTSM/pull/954

===============================================================
===============================================================
Tag name:  ctsm1.0.dev088
Originator(s):  sacks (Bill Sacks)
Date:  Tue Apr  7 10:43:03 MDT 2020
One-line Summary: Fix bug in snow aerosol numerics (causes occasional HUGE aerosol values)

Purpose of changes
------------------

Multiplications / divisions by dtime were missing in a few places, in
some code that ensures that aerosol mass fluxes between the snow layers
don't exceed the total mass. This bug was introduced in ctsm1.0.dev065,
when a term was changed from a mass to a flux (per unit time), but some
of the code wasn't changed correspondingly.

In one case this bug caused a crash, but in other cases it just causes
incorrect answers. The issue can be seen by examining the history
variables SNOBCMCL, SNOOCMCL and SNODSTMCL. In the case that led to the
crash, SNOBCMCL and SNOOCMCL became huge in one grid cell (~ 10^12) for
a long period before the crash.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#965 (Bug in snow aerosol numerics can cause
  aerosol masses to blow up, starting with ctsm1.0.dev065)

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none (not checked, but none expected)

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: Erik Kluzek, Keith Oleson


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    izumi ------- pass

    ok means tests pass, some answer changes as expected

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: all
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      larger than roundoff/same climate

      Answer changes arise due to a bug fix for a condition that only
      appears infrequently. When it does appear, answer changes can
      potentially be enormous (e.g., snow aerosol concentrations 20
      orders of magnitude too large), though answer changes observed in
      the test suite all appear relatively small.

      Answer changes just appeared for the following tests:

        FAIL ERP_D_P36x2_Ld3.f10_f10_musgs.I2000Clm45BgcCrop.cheyenne_gnu.clm-no_subgrid_fluxes BASELINE ctsm1.0.dev087: DIFF
        FAIL SMS_Ld1.nldas2_rnldas2_mnldas2.I2000Ctsm50NwpSpNldasGs.cheyenne_gnu.clm-default BASELINE ctsm1.0.dev087: DIFF
        FAIL SMS_Ld1.nldas2_rnldas2_mnldas2.I2000Ctsm50NwpSpNldasRsGs.cheyenne_gnu.clm-default BASELINE ctsm1.0.dev087: DIFF
        FAIL ERI_D_Ld9.ne30_g16.I2000Clm50BgcCruGs.cheyenne_intel.clm-vrtlay BASELINE ctsm1.0.dev087: DIFF
        FAIL ERI_D_Ld9.T31_g37.I2000Clm50Sp.cheyenne_intel.clm-SNICARFRC BASELINE ctsm1.0.dev087: DIFF
        FAIL ERI_Ld9.f45_g37.I2000Clm50BgcCruGs.cheyenne_intel.clm-nofire BASELINE ctsm1.0.dev087: DIFF
        FAIL ERI_N2_Ld9.f19_g17.I2000Clm50BgcCrop.cheyenne_intel.clm-default BASELINE ctsm1.0.dev087: DIFF
        FAIL SMS_D_Lm6.f45_f45_mg37.I2000Clm50Fates.cheyenne_intel.clm-FatesColdDef BASELINE ctsm1.0.dev087: DIFF
        FAIL SMS_D_Lm6_P144x1.f45_f45_mg37.I2000Clm50Fates.cheyenne_intel.clm-FatesColdDef BASELINE ctsm1.0.dev087: DIFF
        FAIL SMS_Lm13.f19_g17.I2000Clm50BgcCrop.cheyenne_intel.clm-cropMonthOutput BASELINE ctsm1.0.dev087: DIFF

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
https://github.com/ESCOMP/CTSM/pull/971

===============================================================
===============================================================
Tag name:  ctsm1.0.dev087
Originator(s):  sacks (Bill Sacks)
Date:  Mon Apr  6 11:23:51 MDT 2020
One-line Summary: Change hard-coded soil layers in phenology to use a target depth

Purpose of changes
------------------

Adds infrastructure for finding the soil layer containing a given depth
(covered by unit tests).

In phenology code, replaces hard-coded level 3 with a target soil
depth. This value is read from the parameter file. The default value is
set in a way that is bit-for-bit for standard CLM45 and CLM50
configurations, but this will change answers for non-SP configurations
using the NWP soil layer structure or non-standard soil layer
structures.

This is important for the sake of supporting flexible soil layers, as in
the NWP configuration.

Also, some cleanup of the CNPhenology parameter reading code.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#952 (Change hard coded soil layers in phenology
  to use a target depth)


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[X] ctsm5_0-nwp  -- only for non-SP cases

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files):
- New parameter added to NetCDF parameter file: phenology_soil_depth

Substantial timing or memory changes: none expected (not checked)

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: Erik Kluzek


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    izumi ------- pass

    ok means tests pass, a few answer changes as expected

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline:

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: Non-SP cases with NWP or non-standard
      soil layer structures
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      larger than roundoff; same vs. new climate not investigated

      Differences in non-SP cases with soil layer structure different
      from that of standard clm45 or clm50. Differences observed in
      tests
      ERS_D_Ld3.f10_f10_musgs.I2000Clm50BgcCruGs.cheyenne_intel.clm-deepsoil_bedrock
      and
      SOILSTRUCTUD_Ld5.f10_f10_musgs.I2000Clm50BgcCropGs.cheyenne_intel.clm-default.

      ERP_P36x2_D_Ld5.f10_f10_musgs.I2000Ctsm50NwpBgcCropGswpGs.cheyenne_intel.clm-default
      had no differences; I'm guessing this is just because there didn't
      happen to be the appropriate PFTs present to show differences in
      this short, coarse-resolution test.

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
https://github.com/ESCOMP/CTSM/pull/963

===============================================================
===============================================================
Tag name:  ctsm1.0.dev086
Originator(s):  mvertens (Mariana Vertenstein)
Date:  Thu Mar 19 16:03:26 MDT 2020
One-line Summary: Updates to NUOPC cap and minor fixes to ncdio_pio

Purpose of changes
------------------

This tag has updates to the NUOPC cap that are needed for the latest
version of CMEPS.

Now each component NUOPC cap needs to compute possibly time varying
orbital parameters. Before, in cpl7, this was done in the driver and
passed to the components via seq_infodata. However, now the driver is
really just a simple ascii file - so those calculations needed to
migrate to the component caps.

In addition, this tag contains changes to ncdio_pio.F90 that improve
performance in some situations.

This also updates MOSART and RTM tags to include fixes for the NUOPC cap
for MOSART and implementation of an initial NUOPC cap for RTM.

Also, I1Pt compsets now use SROF rather than using an active runoff
model in 'null' mode. This leads to differences in namelists, but not in
results.

Mariana Vertenstein implemented most of the changes in this tag; Jim
Edwards implemented the ncdio_pio changes; Bill Sacks did final testing
and integration.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#943 (NUOPC tests will fail in ctsm1.0.dev085)

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing:
- I1Pt tests and cases now use SROF rather than an active runoff model
  with a null grid (only impacts namelists, not results)
- Changed nuopc test to use MOSART rather than SROF
- Changed finidat file used by LII test to solve a problem that was
  uncovered with the ncdio_pio changes in this tag

Code reviewed by: Bill Sacks


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- pass
    izumi ------- pass

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES (just for NUOPC configurations)

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: Just some NUOPC configurations
    - what platforms/compilers:
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      Not investigated carefully, but likely to be larger than roundoff
     / same climate (due to introduction of time-varying orbital parameters)

     Differences are expected in NUOPC configurations using time-varying
     orbital parameters. Note that no differences showed up in the test
     suite, however, because the only nuopc test in the test suite is
     new in this tag (so did not have baselines).

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.):
- mosart: mosart1_0_35 -> mosart1_0_36
- rtm: rtm1_0_70 -> rtm1_0_71
- cmeps

Pull Requests that document the changes (include PR ids):
https://github.com/ESCOMP/CTSM/issues/943

===============================================================
===============================================================
Tag name:  ctsm1.0.dev085
Originator(s):  sacks (Bill Sacks)
Date:  Mon Mar 16 09:19:57 MDT 2020
One-line Summary: Update cime

Purpose of changes
------------------

Update to latest version of cime master

Bugs fixed or introduced
------------------------

Known bugs introduced in this tag (include github issue ID):
- https://github.com/ESCOMP/CTSM/issues/943 - NUOPC tests will fail in ctsm1.0.dev085


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions):
- NUOPC cases will fail from a standalone checkout

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: self


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    izumi ------- pass

  Two test failures:

    PEND SMS_D_Vnuopc.f10_f10_musgs.I2000Clm50SpRsGs.cheyenne_intel.clm-default SHAREDLIB_BUILD
       I have added this to the expected fails list for now
    FAIL SMS_P720x1_Ln6.hcru_hcru.I2000Clm50BgcCruGs.cheyenne_intel.clm-coldStart BASELINE ctsm1.0.dev084: ERROR BFAIL baseline directory '/glade/p/cgd/tss/ctsm_baselines/ctsm1.0.dev084/SMS_P720x1_Ln6.hcru_hcru.I2000Clm50BgcCruGs.cheyenne_intel.clm-coldStart' does not exist
       This test was added in the previous tag, but apparently wasn't run from there

  Note: I did testing using cime hash
  4f1880756a66ff12d304937d3c166d451ea50cf9, then updated to the tag
  (cime5.8.17). The two are identical except for the cime ChangeLog.

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: NO

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.):
cime: branch_tags/cime5.8.15_a01 -> cime5.8.17

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev084
Originator(s):  negins (Negin Sobhani,UCAR/CSEG,303-497-1224)
Date: Wed Feb 19 15:05:03 MST 2020
One-line Summary: Bounds assertion for C isotopes when threading is on

Purpose of changes
------------------

Previously, the bounds of c13 and c14 in subroutines TruncateCStates and
TruncateCandNStates were not checked when threading was on. This PR allows us
to handle c13 and c14 arrays like everything else in CTSM including (bounds
checking).

This PR resolves issue ESCOMP/ctsm#811


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
	-ESCOMP/ctsm#811

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: 
	- There was an issue (wrong wallclock time ) for one of the tests in the test
	  definitions which got fixed:  (SMS_D_Vnuopc.f10_f10_musgs.I2000Clm50SpRsGs..)
	- Also, I tried making the wallclocktime for all tests in
	  cime_config/testdefs/testlist_clm.xml consistent in the format of hh:mm:ss. 

Code reviewed by: Bill Sacks


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run 

  tools-tests (test/tools):

    cheyenne - not run 

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run 

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run 

  regular tests (aux_clm):

    cheyenne ---- OK 
    izumi ------- PASS

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: NO

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
(https://github.com/ESCOMP/CTSM/pull/923)

===============================================================
===============================================================
Tag name:  ctsm1.0.dev083
Originator(s):  negins (Negin Sobhani,UCAR/CSEG,303-497-1224)
Date: Thu Feb  6 12:12:26 MST 2020
One-line Summary: Some BFB Fixes: Resolve issues #683, # 874, #878, # 885, # 745, #838

Purpose of changes
------------------

Resolved issues: 

ESCOMP/ctsm#683 :  Writing out FATES parameters only on masterproc.
ESCOMP/ctsm#874 :  Changing the number of timesteps to run for
                   SMS_P720x1_Ln3.hcru_hcru.I2000Clm50BgcCruGs.cheyenne_intel.clm-coldStar
                   to prevent failing with CMEPS.
ESCOMP/ctsm#878 :  Removing the unused atm2lnd field.
ESCOMP/ctsm#885 :  Changing the defaults for br_root in namelist_defaults to
                   enable FUN with CLM4.5.
ESCOMP/ctsm#745 :  Removing ReadNL private subroutine from initVerticalMod.F90.
ESCOMP/ctsm#838 :  Clarifying ZISNO in the variable long name.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
Resolves :
	- ESCOMP/ctsm#683
	- ESCOMP/ctsm#874
	- ESCOMP/ctsm#878
	- ESCOMP/ctsm#885
	- ESCOMP/ctsm#745
	- ESCOMP/ctsm#838


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values):
	Not any changes in parameter values.
	changing the default for br_root in namelist_defaults to enable FUN with
	CLM4.5.

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: 
	Changed the number of timesteps from Ln3 to Ln6 for
	SMS_P720x1_Ln3.hcru_hcru.I2000Clm50BgcCruGs.cheyenne_intel.clm-coldStar

Code reviewed by: Bill Sacks


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- OK
    izumi ------- PASS

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: NO

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
https://github.com/ESCOMP/CTSM/pull/897

===============================================================
===============================================================
Tag name:  ctsm1.0.dev082
Originator(s):  oleson (Keith Oleson)
Date:  Sat Feb  1 09:28:41 MST 2020
One-line Summary: Rename variables to avoid confusion; fix QSNOEVAP diagnostic

Purpose of changes
------------------

Renamed variables as discussed in ESCOMP/ctsm#118 throughout the code.

Also made a couple of minor changes to fix a couple of potential
problems with these variables as described in the branch commit logs.

Tested for bfb before changing the history field variables
themselves. These changes are all bfb (with the exception of QSNOEVAP)
for a 1 month global 2deg simulation, but may not be bfb under all
conditions. QSNOEVAP is not bfb because I've added in qflx_ev_snow for
lakes.

Also, update cime slightly with a fix for izumi machine updates

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#118


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none expected (not checked)

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: Bill Sacks


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    izumi ------- ok

    ok: tests pass, answer changes in QSNOEVAP but nothing else (as expected)

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: all
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 

      Larger than roundoff, but only in the diagnostic field QSNOEVAP

      Possibility of other answer changes in rare circumstances, not
      observed in testing.

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.):
- cime: cime5.8.15 -> branch_tags/cime5.8.15_a01
  Minor update for izumi machine changes

Pull Requests that document the changes (include PR ids):
https://github.com/ESCOMP/CTSM/pull/893

===============================================================
===============================================================
Tag name:  ctsm1.0.dev081
Originator(s):  slevis (Samuel Levis,SLevis Consulting LLC,303-665-1310)
Date:  Mon Jan 13 15:37:07 MST 2020
One-line Summary: Speed up restart writes

Purpose of changes
------------------

 E3SM#3163 made some changes to speed up restart writes. I use E3SM#3163
 function GetGlobalIndexArray in place of GetGlobalIndex so as to get
 global indices by array rather than looping over every element.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): #801


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: Limited testing did not show substantial timing changes

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by:
 @billsacks
 @bishtgautam

CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - 

  tools-tests (test/tools):

    cheyenne - 

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - 

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - 

  regular tests (aux_clm):

    cheyenne ---- OK 
    izumi ------- PASS

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline:
 No

 @billsacks recommended running a global test in the branch vs. in master (i.e.
 with vs. without the code mods) to compare restart files because the test suite
 does not compare restart files and an error in the subgrid indices on the
 restart file might go undetected. I ran this test:
 ERS_D_Ld6.f10_f10_musgs.I1850Clm45BgcCrop.cheyenne_intel.clm-clm50CMIP6frc
 and got identical values for all variables in the two restart files.

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
 https://github.com/ESCOMP/ctsm/pull/880

===============================================================
===============================================================
Tag name:  ctsm1.0.dev080
Originator(s):  sacks (Bill Sacks)
Date:  Mon Dec  2 12:23:22 MST 2019
One-line Summary: Update externals, minor fixes to work with latest cime, get nuopc cap working

Purpose of changes
------------------

(1) Update to latest version of externals. This includes a significant
    cime update.

(2) Minor fixes to get things working with the latest version of cime.

(3) Fix from Mariana Vertenstein to get NUOPC cap working, fixes to
    externals cfg files to allow running NUOPC cases out of the box, and
    add a NUOPC test.


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files):
CIME update brings in some changes, e.g., in mapping files; otherwise, none.

Substantial timing or memory changes: not investigated

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing:
- Added NUOPC test:
  SMS_D_Vnuopc.f10_f10_musgs.I2000Clm50SpRsGs.cheyenne_intel.clm-default

Code reviewed by: self


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    izumi ------- ok

    ok means tests passed, some baseline comparisons failed as expected

    Note: Most testing was done on ad1ad752; then, from the latest
    version, I ran a 2-test test list of:
    - SMS_D_Vnuopc.f10_f10_musgs.I2000Clm50SpRsGs.cheyenne_intel.clm-default
    - SMS_D.f10_f10_musgs.I2000Clm50SpRsGs.cheyenne_intel.clm-default

    In addition to verifying that the above tests passed, I also
    inspected the sharedlib build area to verify that separate builds of
    the clm library were done for mct and nuopc.

If the tag used for baseline comparisons was NOT the previous tag, note that here:
- cheyenne tests compared against ctsm1.0.dev078, since cheyenne testing
was not run on ctsm1.0.dev079.


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: Cases at f10 resolution
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      Larger than roundoff, but probably same climate

      Answer changes are due to https://github.com/esmci/cime/pull/3186

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.):
- cime: branch_tags/cime5.8.3_chint17-05 -> cime5.8.15
- rtm: rtm1_0_68 -> rtm1_0_70
- mosart: mosart1_0_34 -> mosart1_0_35
- cmeps: 471ac52 -> 181ff1ed9dfb279e619e8a2173f43baf7bf1dce3
  Also, note that cmeps is now checked out automatically


Pull Requests that document the changes (include PR ids):
https://github.com/ESCOMP/CTSM/pull/845

===============================================================
===============================================================
Tag name:  ctsm1.0.dev079
Originator(s):  sacks (Bill Sacks)
Date:  Mon Nov  4 14:41:41 MST 2019
One-line Summary: Change a few uses of shr_kind

Purpose of changes
------------------

Change a few uses of shr_kind to work with latest cime. These changes
are from Mariana Vertenstein.


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: Bill Sacks


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- NOT RUN!
    izumi ------- pass

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: NO

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev078
Originator(s):  oleson (Keith Oleson)
Date:  Thu Oct 31 12:30:00 MDT 2019
One-line Summary: Fix rootr calculation with use_hydrstress true

Purpose of changes
------------------

In cases with use_hydrstress=.true., both rootr_patch and rootr_col had
problems. This affected the diagnostic fields ROOTR and ROOTR_COLUMN,
and (in BGC cases) fed back on other parts of the system via affecting
the methane code. See https://github.com/ESCOMP/CTSM/issues/812 for more
details.

This tag fixes this issue via these specific changes:

1. Calculate btran in SoilMoistStressMod.F90 regardless of
   use_hydrstress

2. Comment on the applicability of this calculation and that of rootr in
   SoilMoistStressMod.F90

3. Add the following comment to uses of rootr and rootr_col throughout
   the code: "SMS method only"

4. Allow ROOTR_COL history field only if use_hydrstress = .false.

5. Remove the calculation of rootr_col in subroutine
   Compute_EffecRootFrac_And_VertTranSink_HydStress within module
   SoilWaterPlantSinkMod.F90 since that calculation is invalid for PHS.

6. Normalize root resistances for pervious road over nlevgrnd, not
   nlevsoi

Note that the rootr calculation is done using the non-PHS (i.e., SMS)
method. So rootr is not really consistent with PHS; this is something
that should probably be revisited later.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Fixes ESCOMP/ctsm#812 (Problems with rootr when use_hydrstress=.true.)


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: not checked; none expected

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: Bill Sacks


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    izumi ------- ok

    ok means tests pass, answers change as expected

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: Clm50Bgc (cases with both use_hydrstress=.true. and methane active)
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      larger than roundoff / same climate

      For details on differences, see
      https://github.com/ESCOMP/CTSM/issues/812#issuecomment-547924289

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate:
   http://webext.cgd.ucar.edu/I20TR/clm50_ctsm10d074_2deg_GSWP3V1_rootr_hist/lnd/clm50_ctsm10d074_2deg_GSWP3V1_rootr_hist.1995_2014-clm50_ctsm10d074_2deg_GSWP3V1_hist.1995_2014/setsIndex.html
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev077
Originator(s):  sacks (Bill Sacks)
Date:  Sun Oct 27 16:23:39 MDT 2019
One-line Summary: Consistently use frac_sno_eff rather than frac_sno in qg calculations

Purpose of changes
------------------

Consistently use frac_sno_eff rather than frac_sno in qg calculations

The change here only changes answers for (1) urban pervious road, and
(2) more broadly with use_subgrid_fluxes = .false.

Previously, code in CalculateSurfaceHumidity was inconsistent regarding
the use of frac_sno vs. frac_sno_eff. I wanted to make this consistent
to facilitate upcoming cleanup of CalculateSurfaceHumidity. I went back
and forth as to which one we should use here. (See also
https://github.com/ESCOMP/CTSM/issues/822).

Argument for frac_sno_eff: This is consistent with the use of
frac_sno_eff elsewhere in the calculation of surface fluxes.

Argument for frac_sno: Even in CLM4.0 (prior to the introduction of
frac_sno_eff and frac_h2osfc), frac_sno was used to calculate surface
relative humidity. That makes me think that the parameterizations were
improved by using frac_sno here, even though frac_sno at that point was
mainly just used for albedo / radiation calculations. So sticking with
frac_sno would be more consistent with what has been done for a long
time, and would presumably be a bit better scientifically.

However, when I tried changing this to use frac_sno rather than
frac_sno_eff:

    qg(c) = frac_sno_eff(c)*qg_snow(c) + (1._r8 - frac_sno_eff(c) - frac_h2osfc(c))*qg_soil(c) &
          + frac_h2osfc(c) * qg_h2osfc(c)

I got a crash with use_subgrid_fluxes false (test
SMS_D_Ld1_P4x1.f10_f10_musgs.I2000Clm45BgcCropQianRsGs.bishorn_gnu.clm-no_subgrid_fluxes),
due to a check I put in relatively recently to ensure that the top-layer
soil liquid never goes significantly negative:

WJS 218: frac_sno, frac_sno_eff, frac_h2osfc, snl =    6.6447358408778726E-005   1.0000000000000000        0.0000000000000000               -1
 qg_snow, qg_soil, qg_h2osfc, qg =    3.8414454029812475E-003   3.6634834595404343E-003   3.6635176705822736E-003   3.6634952846414733E-003
 ERROR: In UpdateState_TopLayerFluxes, h2osoi_liq has gone significantly negative
 Bulk/tracer name = bulk
 c, lev_top(c) =          218           0
 h2osoi_liq_top_orig  =    6.6448689668604415E-004
 h2osoi_liq           =   -2.6754040213948407E-004
 frac_sno_eff         =    1.0000000000000000
 qflx_liq_grnd*dtime  =    0.0000000000000000
 qflx_dew_grnd*dtime  =    0.0000000000000000
 qflx_evap_grnd*dtime =    9.3202729882552822E-004
 ENDRUN:
 ERROR: In UpdateState_TopLayerFluxes, h2osoi_liq has gone significantly negative

What seems to be going on is: With a little bit of snow present, when I
use frac_sno rather than frac_sno_eff, we end up with a qg value very
close to that of soil. But I think the later evaporation happens from
the snow, at which point I guess the dryer air above leads to there
being too large evaporation from the snow.

So in the end, I decided to consistently use frac_sno_eff, even though
this might be a small step backwards in terms of the accuracy of surface
humidity for urban pervious road and with use_subgrid_fluxes = .false.


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: self (but approved conceptually by Sean Swenson and
Keith Oleson)


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    izumi ------- ok

    ok means tests pass, answers change as expected

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: all
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      Expected to be larger than roundoff/same climate, but not
      investigated carefully

      For the default configuration with use_subgrid_fluxes = .true.,
      differences are restricted to urban pervious road. I spot-checked
      a few tests to ensure that we only saw differences in FSH for
      urban landunits; this was generally true, with the exception of
      ERP_Ly3_P72x2.f10_f10_musgs.IHistClm50BgcCrop.cheyenne_intel.clm-cropMonthOutput,
      which also had differences over crop and natural veg
      landunits. This spillover effect seems to be related to the
      non-local interactions that arise from methane's dependence on
      TWS, though I don't understand why I only saw this in a transient
      case, and not in similar non-transient cases. See
      https://github.com/ESCOMP/CTSM/issues/658#issuecomment-546740771
      for more details.

      For the non-default use_subgrid_fluxes = .false., there
      differences in the natural vegetation and crop landunits, too.

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev076
Originator(s):  sacks (Bill Sacks)
Date:  Fri Oct 25 16:04:20 MDT 2019
One-line Summary: Set frac_sno_eff=0 if frac_sno is 0; avoid unnecessary calls to QSat

Purpose of changes
------------------

Two changes that are bit-for-bit except for changes in the FSNO_EFF
diagnostic field, both with an eye towards cleaning up
SurfaceHumidityMod in preparation for adding water tracers to that
module:

(1) Set frac_sno_eff to 0 if frac_sno is 0. Previously, for cases where
    frac_sno_eff was a binary 0/1 rather than being set equal to
    frac_sno, frac_sno_eff was always being set to 1, even if there was
    no snow. Sean Swenson thinks that didn't matter, but I found it
    confusing, and it could be more problematic with some upcoming
    changes I plan to make (where I'm going to use frac_sno_eff as a
    weighting factor in calculating qg terms, including for landunits
    that use a binary 0/1 frac_sno_eff.)

(2) Avoid unnecessary calls to QSat, since this function is relatively
    expensive.


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: No substantial changes; from a
single run of PFS_Ld20.f09_g17.I2000Clm50BgcCrop.cheyenne_intel, max
timing of the main changed section (bgp1) decreased by about 10%; this
may be within machine variability, though.

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: self


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    izumi ------- ok

    ok means tests pass, answer changes as expected just for FSNO_EFF

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES, but just for the diagnostic field FSNO_EFF

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: all
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      diagnostic only: just changes FSNO_EFF

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev075
Originator(s):  sacks (Bill Sacks)
Date:  Fri Oct 25 15:44:52 MDT 2019
One-line Summary: Lakes: Adjust frac_sno after updating 0-layer snow pack for dew & sublimation

Purpose of changes
------------------

Main change is: for lakes, possibly adjust frac_sno after updating
0-layer snow pack for dew & sub, ensuring that zero vs. non-zero values
of frac_sno agree with whether the snow pack has zero or non-zero
amounts of snow. The main motivation for this is to ensure that, if we
have any snow, then frac_sno should be non-zero. Along with this, I'm
also ensuring that, if we go down to 0 snow, then we also go down to 0
frac_sno.

A related change was to move the code ensuring that h2osno_no_layers
doesn't go negative: I have moved this to before h2osno_no_layers is
referenced, to assure we don't get a slightly negative snow depth. (I
haven't noticed any problems due to the old placement, but it seems
safer to put the limiting before the reference to h2osno_no_layers.)

Also, an unrelated change in run_sys_tests: Add option for rerunning
existing failed tests.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#827 (Lake frac_sno can remain 0 even when there
  is some snow addition from frost)


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: None expected

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: self


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - pass (on my mac: 'make test' with python2 & python3 and 'make lint')

  regular tests (aux_clm):

    cheyenne ---- ok
    izumi ------- ok

    ok means tests pass, answers change as expected

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: all
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      Expected to be larger than roundoff/same climate

      Changes are introduced only over lake columns, and these changes
      should be small, since they only relate to the setting of frac_sno
      when there is dew or sublimation on a very thin snow
      pack.

      Although the direct changes should only affect lakes, I did
      observe changes in crop columns in a transient test that I
      spot-checked
      (ERP_Ly3_P72x2.f10_f10_musgs.IHistClm50BgcCrop.cheyenne_intel.clm-cropMonthOutput)
      - maybe due to some of the non-local interactions in
      https://github.com/ESCOMP/CTSM/wiki/Parameterizations-that-can-cause-nonlocal-interactions,
      or maybe due to some other non-local interaction, either known or
      buggy. (In non-transient tests that I spot checked
      (ERP_P72x2_Lm36.f10_f10_musgs.I2000Clm50BgcCrop.cheyenne_intel.clm-clm50cropIrrigMonth_interp
      and
      SMS_Lm13.f19_g17.I2000Clm50BgcCrop.cheyenne_intel.clm-cropMonthOutput):
      diffs in FSH in h1 file are just over lake landunits, as
      expected.)

         Update: it seems that these changes in crop columns arise from
         the nonlocal interactions caused by the dependence of methane
         on TWS, though I don't understand why I only saw this in a
         transient case, and not in similar non-transient cases. See
         https://github.com/ESCOMP/CTSM/issues/658#issuecomment-546740771
         for more details.

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev074
Originator(s):  sacks (Bill Sacks)
Date:  Wed Oct 23 19:25:21 MDT 2019
One-line Summary: For lakes: when reading finidat, set frac_sno=1 if h2osno_total > 0

Purpose of changes
------------------

Due to ESCOMP/ctsm#783, frac_sno used to be 0 for all lake points. That
was mostly fixed, but the issue is still present on initial conditions
files that were generated prior to that fix (which includes most or all
of our out-of-the-box restart files). This causes frac_sno to be 0 for
lake points at the start of the simulation, even if there is a snow pack
present. Currently this doesn't cause significant problems, but I'd like
to change the calculation of frac_sno_eff so that it is 0 if frac_sno is
0 - and then this frac_sno problem becomes an issue (causing divide by 0
in at least one place).

However, further problems were introduced if I tried to always apply
this correction: It appears that sometimes the restart file legitimately
has frac_sno == 0 for lake columns with non-zero snow cover. To avoid
changing answers for newer restart files, I am writing metadata to the
restart file documenting whether the fix has already been applied on
that restart file, and if so, we avoid reapplying the fix.

Getting this restart metadata correct was tricky, especially when
running init_interp. I have introduced a new module to encapsulate this
complexity. This can be used in general for writing / reading metadata
on which issues have been fixed on a given restart file.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): [If none, remove this line]
- Resolves ESCOMP/ctsm#783


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions):
- The setting of initial lake frac_sno to 1 isn't exactly correct, but
  it's better than the old value of 0.

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none
- I didn't check the PFS test, but the only change that could have
  likely led to performance changes was the addition of a metadata copy
  in init_interp. I checked the timing of this piece, and found it to be
  a negligible portion of the total init_interp time
  (https://github.com/ESCOMP/CTSM/pull/825#issuecomment-545219601).

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: Erik Kluzek gave this an initial look and gave general
approval for the restart file metadata mechanism I have added. He may
give it a more careful look, possibly accompanied by a further cleanup
tag addressing any issues he finds.


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    izumi ------- ok

    ok means tests passed, baselines fail as expected.

    Note that the FUNIT test failed initially; I got it passing on my
    Mac, but didn't rerun it on cheyenne.

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: all
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      Larger than roundoff/same climate

      The only change is in initial lake frac_sno. This should only
      affect the start of the simulation, and only over lake points. I
      verified that changes are only over lake points via: For both
      SMS_D_Ld1.f09_g17.I1850Clm50Sp.cheyenne_intel.clm-default (which
      doesn't use init_interp) and
      SMS_D_Ld3.f10_f10_musgs.I2000Clm50BgcCruGs.cheyenne_gnu.clm-default
      (which uses init_interp): I checked which points have differences
      in FSH in the h1 (vector) history file. Diffs were only over lake
      points.

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
https://github.com/ESCOMP/CTSM/pull/825

===============================================================
===============================================================
Tag name:  ctsm1.0.dev073
Originator(s):  sacks (Bill Sacks)
Date:  Tue Oct 22 06:14:24 MDT 2019
One-line Summary: Fix bug in calculation of dqgdT

Purpose of changes
------------------

The calculation of dqgdT, which gives the derivative, d(qg)/dT, was
incorrect for soil/crop landunits when there is no snow. This tag fixes
that bug.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/CTSM#824 (dqgdT incorrect when there is no snow)


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: not checked, but none expected

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: Sean Swenson reviewed and approved my plan, though not the final code


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    izumi ------- ok

    ok means tests pass, answers change as expected

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: all
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      Larger than roundoff/same climate

      I checked the impact from one-day averages in test
      SMS_D_Ld1_P4x1.f10_f10_musgs.I2000Clm50BgcCropQianRsGs.bishorn_gnu.clm-default,
      by looking at differences in EFLX_LH_TOT. The impact is small:
      daily-average differences in that variable are typically 0.01 W
      m-2 or less; most grid cells have significantly less difference
      than that; only one or a few f10 grid cells have significantly
      greater differences, with the biggest difference being about 0.1 W
      m-2.

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev072
Originator(s):  mvertens (Mariana Vertenstein)
Date:  Tue Oct 15 09:41:50 MDT 2019
One-line Summary: Add NUOPC cap

Purpose of changes
------------------

Add NUOPC cap in order to run with the new NUOPC driver/mediator. There
was some code movement / refactoring of existing code to support this,
with the biggest change being the introduction of a new subdirectory
under src/cpl.

Most changes are from Mariana Vertenstein and Jim Edwards.

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none expected (not checked)

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance):
- NUOPC cap currently does not have any tests in the aux_clm test suite
- There are currently no tests ensuring that the nuopc and mct caps give
  bit-for-bit answers, though the nuopc_cap_bfb testmod could be used to
  support such a test

Changes to tests or testing: none

Code reviewed by: Bill Sacks


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- pass
    izumi ------- pass

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: NO

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.):
- mosart: mosart1_0_33 -> mosart1_0_34 (supports nuopc cap)

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev071
Originator(s):  sacks (Bill Sacks)
Date:  Fri Oct 11 07:03:40 MDT 2019
One-line Summary: Split CanopyTemperature into separate pieces

Purpose of changes
------------------

CanopyTemperature was doing a few unrelated things, none of which
actually had anything to do with calculating canopy temperature. At the
driver level, I have split CanopyTemperature into BiogeophysPreFluxCalcs
and CalculateSurfaceHumidity. BiogeophysPreFluxCalcs calls out to a
number of separate routines, some in that same module and some
elsewhere, including a new routine in FrictionVelocityMod.

Also some minor related cleanup, including

- Made FrictionVelocityMod object-oriented

- A bit of rework of how qflx_evap_veg and qflx_tran_veg are set. This
  changes answers for the (default inactive) QFLX_EVAP_VEG history field
  (see also ESCOMP/ctsm#816).

- Removed qsat conditionals that should never be invoked


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Fixes ESCOMP/ctsm#816 (QFLX_EVAP_VEG should be 0 over lakes)


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: self


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- pass
    izumi ------- pass

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES, but only for the
(default-inactive) field QFLX_EVAP_VEG

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: all
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      larger than roundoff, but only for the default-inactive field QFLX_EVAP_VEG

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev070
Originator(s):  sacks (Bill Sacks)
Date:  Wed Oct  9 06:16:39 MDT 2019
One-line Summary: Fix for writing 0-d variables with PIO2

Purpose of changes
------------------

Previously, there were problems when writing 0-d (scalar) variables with
PIO2 in DEBUG mode. This change fixes that issue.

Jim Edwards put in place an initial fix; I put in place the final fix
based on changes Mariana Vertenstein made in MOSART. I have not tested
this with PIO2, but am hopeful that it fixes the PIO2 problem since the
similar MOSART changes fixed that problem.

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: self; earlier version reviewed by Erik Kluzek and Mariana Vertenstein


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- pass
    izumi ------- pass

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: NO

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
https://github.com/ESCOMP/CTSM/pull/810

===============================================================
===============================================================
Tag name:  ctsm1.0.dev069
Originator(s):  sacks (Bill Sacks)
Date:  Mon Oct  7 10:22:22 MDT 2019
One-line Summary: Misc. code cleanup and minor bug fixes

Purpose of changes
------------------

Resolve a variety of "simple bfb" issues. Note that some of these result
in answer changes for select diagnostic fields. See specific notes below
for details, under "Issues fixed" and answer changes.

The fix for ESCOMP/ctsm#27 changes answers for some diagnostic fields,
as noted below.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#27 (BTRAN should be spval over all special
  landunits)
  - Also fixed some related fields, as noted below, under answer changes

- Resolves ESCOMP/ctsm#31 (some CLM history fields don't restart
  properly)
  - This was mostly resolved previously; here I just did some final
    cleanup and fixing of NFIRE

- Resolves ESCOMP/ctsm#48 (code for initializing some isotope carbon
  state variables looks wrong)
  - Removed the incorrect code, which was never being exercised

- Resolves ESCOMP/ctsm#58 (Change SHR_ASSERT calls to avoid using
  errMsg)
  - Purpose was to speed up DEBUG runs on some machines/compilers

- Resolves ESCOMP/ctsm#111 (Precision of constants in
  SoilTemperature.F90 is not double)
  - No answer changes, because 1 (single prec) = 1 (double prec)

- Resolves ESCOMP/ctsm#334 (Introduce a new activelayer_type)

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: potentially speeds up debug runs on some machines/compilers

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: self


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    izumi ------- ok

    ok means tests pass, some baseline comparisons fail as expected

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES, but just for select diagnostic fields

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: ALL
    - what platforms/compilers: ALL
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      Greater than roundoff, but only in select diagnostic field - no impact on climate

      There were changes in the following diagnostic fields, which are
      now averaged over only vegetated landunits (see ESCOMP/ctsm#27):
      - BTRAN
      - BTRANMN
      - RSSHA
      - RSSUN
      - ROOTR (inactive by default)
      - ROOTR_COLUMN (inactive by default)
      - RRESIS (inactive by default)

      Changes are also possible in rare circumstances for the NFIRE
      diagnostic field, in transient cases, although no differences were
      observed for this field in the test suite (see ESCOMP/ctsm#31).

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev068
Originator(s):  sacks (Bill Sacks)
Date:  Mon Sep 30 09:57:04 MDT 2019
One-line Summary: Add water tracers to CombineSnowLayers, DivideSnowLayers, ZeroEmptySnowLayers

Purpose of changes
------------------

Add water tracers to CombineSnowLayers, DivideSnowLayers and
ZeroEmptySnowLayers. This completes the tracerization of the core snow
code: most or all of the code in SnowHydrologyMod is now tracerized.

In contrast to most routines that I have tracerized, the tracerization
of these routines is done by embedding loops over tracers inside
existing code structures, rather than splitting out separate routines
that operate on bulk or tracer quantities. This is because (1) it didn't
make sense to introduce explicit fluxes for the state updates done here,
so we didn't have the extra need of having separate routines for flux
calculations vs. state updates; (2) the state updates were embedded in
loops over levels and conditionals in a way that would have been awkward
and messy to try to separately update bulk-and-tracer quantities
(h2osoi_liq, h2osoi_ice, etc.) from bulk-only quantities (dz, etc.).

Also, a bit of reordering of LakeHydrologyMod in order to keep the snow
code more together.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#778 (Implement water tracers for combining and
  dividing snow layers)

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none 

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: I don't think so
   Based on the changes introduced, I expect *very* small increases in
   runtime of the modified routines (which would show up in the
   hydro_without_drainage timer), even for cases without water tracers,
   but I wouldn't expect this to have a significant impact. From PFS
   test, and specifically looking at the timing of
   hydro_without_drainage in that test, I see a possible small increase
   (a few percent) in the timing of that section, though the differences
   are within typical machine variability, so don't seem necessarily
   significant.

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: self


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- pass
    izumi ------- pass

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: NO

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev067
Originator(s):  erik (Erik Kluzek,UCAR/TSS,303-497-1326)
Date: Thu Sep 26 15:02:39 MDT 2019
One-line Summary: Add config_archive.xml to CTSM cime_config -- no testing

Purpose of changes
------------------

Add config_archive.xml to cime_config for CTSM. Update the CODE_OF_CONDUCT
to the August/2019 UCAR version.

No testing done. 

Bugs fixed or introduced
------------------------

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): None

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): None

Changes made to namelist defaults (e.g., changed parameter values): None

Changes to the datasets (e.g., parameter, surface or initial files): None

Substantial timing or memory changes: None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): None

Changes to tests or testing: None

Code reviewed by: self


CTSM testing: doc (no standard testing done)

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS

  src unit-tests:

    cheyenne - PASS

  regular tests (aux_clm): None

If the tag used for baseline comparisons was NOT the previous tag, note that here: previous


Answer changes
--------------

Changes answers relative to baseline: No!

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): None

Pull Requests that document the changes (include PR ids): None
(https://github.com/ESCOMP/ctsm/pull)

===============================================================
===============================================================
Tag name:  ctsm1.0.dev066
Originator(s):  sacks (Bill Sacks)
Date: Fri Sep 20 09:07:23 MDT 2019
One-line Summary: Add water tracers to SnowCapping

Purpose of changes
------------------

Update water tracer terms in SnowCapping. As I have done elsewhere, this
has involved splitting each piece of SnowCapping into its own subroutine,
which operates either on bulk, tracers, or bulk + tracers.

This is supported by calls to ResetCheckedTracers prior to the call to
SnowCapping, since some intervening code is not yet tracerized.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#777 (Implement water tracers for SnowCapping)


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none
   Small increase in total runtime of PFS test, but the relevant timing
   section, hydro_without_drainage, took slightly *less* time compared
   with the last tag.


Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: self


CTSM testing:

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- pass
    izumi ------- pass

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: NO


Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev065
Originator(s):  sacks (Bill Sacks)
Date: Sat Sep 14 12:52:42 MDT 2019
One-line Summary: Add water tracers to SnowWater

Purpose of changes
------------------

Update water tracer terms in SnowWater. As I have done elsewhere, this
has involved splitting each piece of SnowWater into its own subroutine,
which operates either on bulk, tracers, or bulk + tracers.

This is supported by calls to ResetCheckedTracers prior to the call to
SnowWater, since some intervening code is not yet tracerized.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#776 (Implement water tracers for SnowWater)


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: self


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    izumi ------- ok

    ok means tests pass, baselines fail as expected

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: all
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 

      Roundoff level: answer changes just arise because I have made
      qflx_snow_percolation_col a true flux, with units of
      per-unit-time; this involved dividing by dtime, and later
      multiplying by dtime when it is applied to states. Previously, the
      corresponding variable (qout / qin) was not per-unit-time.

   If bitwise differences were observed, how did you show they were no worse
   than roundoff?

   I ran comparisons against a version off of master that had the
   minimal changes to exhibit these roundoff-level changes. Diffs for
   this are below. My branch is bit-for-bit with this version.

        diff --git a/src/biogeophys/SnowHydrologyMod.F90 b/src/biogeophys/SnowHydrologyMod.F90
        index cad47e28..0abcc49f 100644
        --- a/src/biogeophys/SnowHydrologyMod.F90
        +++ b/src/biogeophys/SnowHydrologyMod.F90
        @@ -1140,17 +1140,17 @@ subroutine SnowWater(bounds, &
                   c = filter_snowc(fc)
                   if (j >= snl(c)+1) then
         
        -             h2osoi_liq(c,j) = h2osoi_liq(c,j) + qin(c)
        +             h2osoi_liq(c,j) = h2osoi_liq(c,j) + qin(c)*dtime
         
        -             mss_bcphi(c,j) = mss_bcphi(c,j) + qin_bc_phi(c)
        -             mss_bcpho(c,j) = mss_bcpho(c,j) + qin_bc_pho(c)
        -             mss_ocphi(c,j) = mss_ocphi(c,j) + qin_oc_phi(c)
        -             mss_ocpho(c,j) = mss_ocpho(c,j) + qin_oc_pho(c)
        +             mss_bcphi(c,j) = mss_bcphi(c,j) + qin_bc_phi(c)*dtime
        +             mss_bcpho(c,j) = mss_bcpho(c,j) + qin_bc_pho(c)*dtime
        +             mss_ocphi(c,j) = mss_ocphi(c,j) + qin_oc_phi(c)*dtime
        +             mss_ocpho(c,j) = mss_ocpho(c,j) + qin_oc_pho(c)*dtime
         
        -             mss_dst1(c,j)  = mss_dst1(c,j) + qin_dst1(c)
        -             mss_dst2(c,j)  = mss_dst2(c,j) + qin_dst2(c)
        -             mss_dst3(c,j)  = mss_dst3(c,j) + qin_dst3(c)
        -             mss_dst4(c,j)  = mss_dst4(c,j) + qin_dst4(c)
        +             mss_dst1(c,j)  = mss_dst1(c,j) + qin_dst1(c)*dtime
        +             mss_dst2(c,j)  = mss_dst2(c,j) + qin_dst2(c)*dtime
        +             mss_dst3(c,j)  = mss_dst3(c,j) + qin_dst3(c)*dtime
        +             mss_dst4(c,j)  = mss_dst4(c,j) + qin_dst4(c)*dtime
         
                      if (j <= -1) then
                         ! No runoff over snow surface, just ponding on surface
        @@ -1167,8 +1167,8 @@ subroutine SnowWater(bounds, &
                         qout(c) = max(0._r8,(vol_liq(c,j) &
                              - ssi*eff_porosity(c,j))*dz(c,j)*frac_sno_eff(c))
                      end if
        -             qout(c) = qout(c)*1000._r8
        -             h2osoi_liq(c,j) = h2osoi_liq(c,j) - qout(c)
        +             qout(c) = (qout(c)*1000._r8)/dtime
        +             h2osoi_liq(c,j) = h2osoi_liq(c,j) - qout(c)*dtime
                      qin(c) = qout(c)
         
                      ! mass of ice+water: in extremely rare circumstances, this can
        @@ -1187,7 +1187,7 @@ subroutine SnowWater(bounds, &
                      if (qout_bc_phi(c) > mss_bcphi(c,j)) then
                         qout_bc_phi(c) = mss_bcphi(c,j)
                      endif
        -             mss_bcphi(c,j) = mss_bcphi(c,j) - qout_bc_phi(c)
        +             mss_bcphi(c,j) = mss_bcphi(c,j) - qout_bc_phi(c)*dtime
                      qin_bc_phi(c) = qout_bc_phi(c)
         
                      ! BCPHO:
        @@ -1196,7 +1196,7 @@ subroutine SnowWater(bounds, &
                      if (qout_bc_pho(c) > mss_bcpho(c,j)) then
                         qout_bc_pho(c) = mss_bcpho(c,j)
                      endif
        -             mss_bcpho(c,j) = mss_bcpho(c,j) - qout_bc_pho(c)
        +             mss_bcpho(c,j) = mss_bcpho(c,j) - qout_bc_pho(c)*dtime
                      qin_bc_pho(c) = qout_bc_pho(c)
         
                      ! OCPHI:
        @@ -1205,7 +1205,7 @@ subroutine SnowWater(bounds, &
                      if (qout_oc_phi(c) > mss_ocphi(c,j)) then
                         qout_oc_phi(c) = mss_ocphi(c,j)
                      endif
        -             mss_ocphi(c,j) = mss_ocphi(c,j) - qout_oc_phi(c)
        +             mss_ocphi(c,j) = mss_ocphi(c,j) - qout_oc_phi(c)*dtime
                      qin_oc_phi(c) = qout_oc_phi(c)
         
                      ! OCPHO:
        @@ -1214,7 +1214,7 @@ subroutine SnowWater(bounds, &
                      if (qout_oc_pho(c) > mss_ocpho(c,j)) then
                         qout_oc_pho(c) = mss_ocpho(c,j)
                      endif
        -             mss_ocpho(c,j) = mss_ocpho(c,j) - qout_oc_pho(c)
        +             mss_ocpho(c,j) = mss_ocpho(c,j) - qout_oc_pho(c)*dtime
                      qin_oc_pho(c) = qout_oc_pho(c)
         
                      ! DUST 1:
        @@ -1223,7 +1223,7 @@ subroutine SnowWater(bounds, &
                      if (qout_dst1(c) > mss_dst1(c,j)) then
                         qout_dst1(c) = mss_dst1(c,j)
                      endif
        -             mss_dst1(c,j) = mss_dst1(c,j) - qout_dst1(c)
        +             mss_dst1(c,j) = mss_dst1(c,j) - qout_dst1(c)*dtime
                      qin_dst1(c) = qout_dst1(c)
         
                      ! DUST 2:
        @@ -1232,7 +1232,7 @@ subroutine SnowWater(bounds, &
                      if (qout_dst2(c) > mss_dst2(c,j)) then
                         qout_dst2(c) = mss_dst2(c,j)
                      endif
        -             mss_dst2(c,j) = mss_dst2(c,j) - qout_dst2(c)
        +             mss_dst2(c,j) = mss_dst2(c,j) - qout_dst2(c)*dtime
                      qin_dst2(c) = qout_dst2(c)
         
                      ! DUST 3:
        @@ -1241,7 +1241,7 @@ subroutine SnowWater(bounds, &
                      if (qout_dst3(c) > mss_dst3(c,j)) then
                         qout_dst3(c) = mss_dst3(c,j)
                      endif
        -             mss_dst3(c,j) = mss_dst3(c,j) - qout_dst3(c)
        +             mss_dst3(c,j) = mss_dst3(c,j) - qout_dst3(c)*dtime
                      qin_dst3(c) = qout_dst3(c)
         
                      ! DUST 4:
        @@ -1250,7 +1250,7 @@ subroutine SnowWater(bounds, &
                      if (qout_dst4(c) > mss_dst4(c,j)) then
                         qout_dst4(c) = mss_dst4(c,j)
                      endif
        -             mss_dst4(c,j) = mss_dst4(c,j) - qout_dst4(c)
        +             mss_dst4(c,j) = mss_dst4(c,j) - qout_dst4(c)*dtime
                      qin_dst4(c) = qout_dst4(c)
         
                   end if
        @@ -1280,9 +1280,9 @@ subroutine SnowWater(bounds, &
             do fc = 1, num_snowc
                c = filter_snowc(fc)
                ! Qout from snow bottom
        -       qflx_snow_drain(c) = qflx_snow_drain(c) + (qout(c) / dtime)
        +       qflx_snow_drain(c) = qflx_snow_drain(c) + qout(c)
         
        -       qflx_rain_plus_snomelt(c) = (qout(c) / dtime) &
        +       qflx_rain_plus_snomelt(c) = qout(c) &
                     + (1.0_r8 - frac_sno_eff(c)) * qflx_liq_grnd(c)
                int_snow(c) = int_snow(c) + frac_sno_eff(c) &
                              * (qflx_dew_snow(c) + qflx_dew_grnd(c) + qflx_liq_grnd(c)) * dtime



   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev064
Originator(s):  slevis (Samuel Levis,SLevis Consulting LLC,303-665-1310)
Date:  Mon Sep  9 13:14:51 MDT 2019
One-line Summary: User defined top-two snow layers

Purpose of changes
------------------

 Instead of hardcoding, we now set:

 dzmin(1) = snow_dzmin_1
 dzmax_l(1) = snow_dzmax_l_1
 dzmax_u(1) = snow_dzmax_u_1
 dzmin(2) = snow_dzmin_2
 dzmax_l(2) = snow_dzmax_l_2
 dzmax_u(2) = snow_dzmax_u_2

 where the right-hand-side variables get namelist-defined values and the
 model calculates dzmin and dzmax_* for the remaining snow layers using
 recursive formulas. The calculation is intended to match the previously
 hardcoded values, except for the bottom snow layer when nlevsno < 12.
 The bottom layer now always gets dzmax_* = huge(1._r8). The formulas
 appear in a document titled "A firn model for CLM" that is linked to
 #729.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): #729


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables):
 New namelist parameters for specifying the top-two snow layers:
 snow_dzmin_1, snow_dzmax_l_1, snow_dzmax_u_1
 snow_dzmin_2, snow_dzmax_l_2, snow_dzmax_u_2

Changes made to namelist defaults (e.g., changed parameter values):
 Introduced default values for the new parameters so as to maintain
 same answers to within roundoff

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: @billsacks


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - 

  tools-tests (test/tools):

    cheyenne - 

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - 

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - 

  regular tests (aux_clm):

    cheyenne ---- OK
    izumi ------- OK

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES

  Summarize any changes to answers, i.e.,
    - what code configurations: all
    - what platforms/compilers: all
    - nature of change: roundoff

   If bitwise differences were observed, how did you show they were no worse
   than roundoff?
 Introduced temporary code to overwrite dzmin(j), dzmax_l(j), and
 dzmax_u(j) with their original values when different by less than
 1e-13 from their original values for all but the bottom snow layer.
 The bottom layer was left as dzmax_l = dzmax_u(j) = huge(1._8), i.e.
 the new value. The test resulted in bit-for-bit same answers.

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
 https://github.com/ESCOMP/ctsm/pull/792

===============================================================
===============================================================
Tag name:  ctsm1.0.dev063
Originator(s):  erik (Erik Kluzek)
Date: Thu Sep  5 21:28:02 MDT 2019
One-line Summary: Two answer changing fixes (fire, DWT_SLASH) and fix for urban streams for Clm45

Purpose of changes
------------------

Fire bug fix and DWT_SLASH change and fix urban stream years for clm4_5.
The latitude used in an expression for Bgc-Fire model Li2016 (used in Clm50)
had latitude in degrees rather than in radians as it should have been. The
term is used for ignition and the cosine was being taken in degrees rather 
than radians, hence the spatial pattern for ignition with latitude was incorrect.

The history field DWT_SLASH_CFLUX was being calculated on the column, but 
really should have been a patch level variable. In the same way other fields
were handled we made DWT_SLASH_CFLUX a grid-cell variable, and added
DWT_SLASH_CFLUX_PATCH on the patch level that could be added to output. The same
is true of the C13_ and C14_ versions of it.

Clm45 was setting urbantv to year 2000, rather than 1850 or historical as it should
have. The urbantv file for Clm45 doesn't actually change, so this doesn't actually
make a difference. But, it does make the namelist look wrong.

Note, that SP cases, Vic cases and Fates cases do NOT change answers. Or if fire is
off, or certain single-point cases. Some cold-start cases don't seem to be sensitive
to it either.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): #175, #787, #780
  fixes #175 urban stream years for clm4_5
  fixes #787 DWT_SLASH field
  fixes #780 Bug in CN Fire Li 2016 which used latitude in degrees rather than radians

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[X] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): None

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): None

Changes made to namelist defaults (e.g., changed parameter values): Defaults for urbantv for clm4_5

Changes to the datasets (e.g., parameter, surface or initial files): None

Substantial timing or memory changes: None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): None

Changes to tests or testing: None

Code reviewed by: self, @olywon


CTSM testing: regular

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS (62 compare fail due to namelist changes)

  regular tests (aux_clm):

    cheyenne ---- OK
    izumi ------- OK

If the tag used for baseline comparisons was NOT the previous tag, note that here: previous


Answer changes
--------------

Changes answers relative to baseline: Yes!

  Summarize any changes to answers, i.e.,
    - what code configurations: Clm50Bgc and some Clm45Bgc cases
    - what platforms/compilers: All
    - nature of change: similar climate (but fire ignition pattern changes by latitude)
        DWT_SLASH_* history fields change because moved to gridcell quantity (so change is relatively minor)

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: 
           oleson/clm50_cesm20R_2deg_GSWP3V1_issue780_hist (for the fire change)

   URL for LMWG diagnostics output used to validate new climate:
http://webext.cgd.ucar.edu/I20TR/clm50_cesm20R_2deg_GSWP3V1_issue780_hist/lnd/clm50_cesm20R_2deg_GSWP3V1_issue780_hist.1995_2014-clm50_cesm20R_2deg_GSWP3V1_hist.1995_2014/setsIndex.html
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): None

Pull Requests that document the changes (include PR ids): #802
(https://github.com/ESCOMP/ctsm/pull)
   #802 -- Three answer changing fixes (fire, DWT_SLASH, urban streams for clm4_5)

===============================================================
===============================================================
Tag name:  ctsm1.0.dev062
Originator(s):  sacks (Bill Sacks)
Date: Tue Sep  3 16:04:28 MDT 2019
One-line Summary: Move hobart tests to izumi

Purpose of changes
------------------

Move all hobart testing to izumi.


Bugs fixed or introduced
------------------------

Known bugs introduced in this tag (include github issue ID):
- Recurrence of ESCOMP/ctsm#174 (ne30 case fails on hobart_nag due to floating overflow)

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance):
- Testing should now be run on izumi, not hobart

Changes to tests or testing:
- All hobart tests moved to izumi

Code reviewed by: self


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- not run
    izumi ------- pass

  Also ran prealpha and prebeta tests that I moved from hobart to izumi. These passed except:
  - ERP_D_Ld5.ne30_g16.I1850Clm50BgcCrop.izumi_nag.clm-default (#174)
  - ERP_Ld5.f19_g17.I2000Clm50SpRtmFl.izumi_pgi.clm-default
  - SMS_D_Ld1.f19_g17.I1850Clm45Cn.izumi_pgi.clm-default

  The last two failed with errors like:

    i011.unified.ucar.edu.44619hfi_userinit: mmap of status page (dabbad0008040000) failed: Operation not permitted

  I wonder if the issue is that multi-node pgi cases don't work on izumi?

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: NO

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.):
- cime: cime5.8.3_chint17-04 -> cime5.8.3_chint17-05
  Minor change on izumi to allow using run_sys_tests

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev061
Originator(s):  erik (Erik Kluzek)
Date: Sun Sep  1 22:37:07 MDT 2019
One-line Summary: Simple b4b fixes: new params file, remove override_nsrest/anoxia_wtsat, DV deprecated

Purpose of changes
------------------

New clm5 paramsfile that has the bad date of 631 changed to 701. Along with some simple bit for
bit changes. Removing some problematic options. Making Dynamic Vegetation a deprecated option 
that requires you to use -ignore_warnings with it.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):  #463 #79 #97 #98 #104 #122 #173 #794 #795 #796

   Fixes #463 bad date on params file that didn't allow using ESMF library with Crop
   Fixes #79 correct documentation of qflx_evap_tot
   Fixes #97 remove override_nsrest
   Fixes #98 add warning when using single instance of a startup file for multi-instance cases
   Fixes #104 use get_step_size_real when putting into a real variable
   Fixes #122 decomp pool arrays should not start at 0 but 1
   Fixes #173 remove psi_soil_ref as not used
   Fixes #794 remove anoxia_wtsat
   Fixes #795 Correct documentation of -light_res
   Fixes #796 Deprecate Dynamic Vegetation

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): None

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): Remove namelist items
   Remove namelist items:  anoxia_wtsat, override_nsrest
   Turning dynamic Vegetation on, now displays a warning, and requires using -ignore_warnings in CLM_BLDNML_OPTS

Changes made to namelist defaults (e.g., changed parameter values): New paramdata files

Changes to the datasets (e.g., parameter, surface or initial files): 
   clm5 paramdata file has correct date for crop of 701 (rather than incorrect 631)
   clm5 and clm4_5 paramdata file removed an unused variable

Substantial timing or memory changes: None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): None

Changes to tests or testing: DV tests removed or changed

Code reviewed by: self

CTSM testing: regular


 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS (compare shows changes becausse of paramdata files)

  regular tests (aux_clm):

    cheyenne ---- OK
    hobart ------ OK

An additional fail is ERP_P36x2_D_Ld5.f10_f10_musgs.I2000Clm50Cn.cheyenne_gnu.clm-default, because
of #798. Because it's an issue with cime, I'm not marking it as an expected error.

If the tag used for baseline comparisons was NOT the previous tag, note that here: previous


Answer changes
--------------

Changes answers relative to baseline: No bit-for-bit

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): None

Pull Requests that document the changes (include PR ids): #797
(https://github.com/ESCOMP/ctsm/pull)

  #797 -- Simple b4b fixes: new params file, remove override_nsrest/anoxia_wtsat, DV deprecated 

===============================================================
===============================================================
Tag name:  ctsm1.0.dev060
Originator(s):  sacks (Bill Sacks)
Date:  Thu Aug 29 11:18:20 MDT 2019
One-line Summary: In SnowWater, truncate small h2osoi residuals

Purpose of changes
------------------

Previously, in the handling of sublimation and evaporation from the snow
pack, if these fluxes led to negative h2osoi_ice or h2osoi_liq, these
negative values would be passed down the snow pack until they found a
layer with a large enough positive state to absorb them. This logic was
going to be a challenge for adding water tracers.

I have done some tests to verify that these negative values never exceed
roundoff-level in either an absolute or relative sense (with tolerance
of 1.e-13). Specifically, I ran the full aux_clm test suite with the
diffs below and verified that the endrun calls were never triggered.

So here, I simplify this logic: After truncating any near-zero states to
exactly zero, I check to confirm that the states are never negative. I
then removed the code that passed these negative values down the snow
pack: if they are only roundoff-level negative, then it seems safe to
just throw them away rather than trying to handle them.

Here were the diffs used to verify that the negative values are never
greater than roundoff-level; these endrun calls were never triggered in
a run of the full aux_clm test suite:

    diff --git a/src/biogeophys/SnowHydrologyMod.F90 b/src/biogeophys/SnowHydrologyMod.F90
    index 3dd555cc..07776123 100644
    --- a/src/biogeophys/SnowHydrologyMod.F90
    +++ b/src/biogeophys/SnowHydrologyMod.F90
    @@ -970,6 +970,8 @@ subroutine SnowWater(bounds, &
         real(r8) :: vol_ice(bounds%begc:bounds%endc,-nlevsno+1:0)      ! partial volume of ice lens in layer
         real(r8) :: eff_porosity(bounds%begc:bounds%endc,-nlevsno+1:0) ! effective porosity = porosity - vol_ice
         real(r8) :: mss_liqice(bounds%begc:bounds%endc,-nlevsno+1:0)   ! mass of liquid+ice in a layer
    +    real(r8) :: h2osoi_ice_orig
    +    real(r8) :: h2osoi_liq_orig
         !-----------------------------------------------------------------------

         associate( &
    @@ -1017,10 +1019,23 @@ subroutine SnowWater(bounds, &
            c = filter_snowc(fc)
            l=col%landunit(c)

    +       h2osoi_ice_orig = h2osoi_ice(c,snl(c)+1)
    +       h2osoi_liq_orig = h2osoi_liq(c,snl(c)+1)
    +
            wgdif = h2osoi_ice(c,snl(c)+1) &
                 + frac_sno_eff(c) * (qflx_dew_snow(c) - qflx_sub_snow(c)) * dtime
            h2osoi_ice(c,snl(c)+1) = wgdif
            if (wgdif < 0._r8) then
    +          if (wgdif < -1.e-13_r8) then
    +             write(iulog,*) 'WJS: big abs wgdif ice: ', c, wgdif, h2osoi_ice_orig
    +             call endrun('WJS: big abs wgdif ice')
    +          end if
    +          if (h2osoi_ice_orig > 0._r8) then
    +             if (abs(wgdif / h2osoi_ice_orig) > 1.e-13_r8) then
    +                write(iulog,*) 'WJS: big rel wgdif ice: ', c, wgdif, h2osoi_ice_orig
    +                call endrun('WJS: big rel wgdif ice')
    +             end if
    +          end if
               h2osoi_ice(c,snl(c)+1) = 0._r8
               h2osoi_liq(c,snl(c)+1) = h2osoi_liq(c,snl(c)+1) + wgdif
            end if
    @@ -1030,6 +1045,16 @@ subroutine SnowWater(bounds, &

            ! if negative, reduce deeper layer's liquid water content sequentially
            if(h2osoi_liq(c,snl(c)+1) < 0._r8) then
    +          if (h2osoi_liq(c,snl(c)+1) < -1.e-13_r8) then
    +             write(iulog,*) 'WJS: big abs wgdif liq: ', c, h2osoi_liq(c,snl(c)+1), h2osoi_liq_orig
    +             call endrun('WJS: big abs wgdif liq')
    +          end if
    +          if (h2osoi_liq_orig > 0._r8) then
    +             if (abs(h2osoi_liq(c,snl(c)+1) / h2osoi_liq_orig) > 1.e-13_r8) then
    +                write(iulog,*) 'WJS: big rel wgdif liq: ', c, h2osoi_liq(c,snl(c)+1), h2osoi_liq_orig
    +                call endrun('WJS: big rel wgdif liq')
    +             end if
    +          end if
               do j = snl(c)+1, 1
                  wgdif=h2osoi_liq(c,j)
                  if (wgdif >= 0._r8) exit

Bugs fixed or introduced
------------------------

None

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions):
- It's possible that the extra error checks I have added (to ensure we
  don't have greater-than-roundoff-level negative residuals) will be
  triggered in rare circumstances in a production run, even though they
  were never triggered in the test suite.

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none
Although there was a very small increase in total lnd run time, the
timing of hydro_without_drainage actually decreased slightly in the PFS
test relative to the last tag.

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance):
- It's possible that the extra error checks I have added (to ensure we
  don't have greater-than-roundoff-level negative residuals) will be
  triggered in rare circumstances in a production run, even though they
  were never triggered in the test suite.

Changes to tests or testing: none

Code reviewed by: self


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    hobart ------ ok

    ok means tests passed, some answer changes as expected

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: potentially all, though only show up in
      a few tests
    - what platforms/compilers: potentially all, though only show up in
      a few tests
    - nature of change (roundoff; larger than roundoff/same climate; new climate): roundoff

   If bitwise differences were observed, how did you show they were no worse
   than roundoff?

      First, I did some preliminary analyses as documented above, to
      ensure that any negative residuals were no greater than roundoff.

      Second, note that the use of truncate_small_values can only
      introduce roundoff-level changes (by truncating values that are
      roundoff-level different from zero to exactly zero).

      Third, only a few tests showed answer changes, and these were only
      in limited fields, and were fairly small RMS differences:

         ERP_Ly3_P72x2.f10_f10_musgs.IHistClm50BgcCrop.cheyenne_intel.clm-cropMonthOutput
         ERS_Ly3.f10_f10_musgs.I1850Clm50BgcCropCmip6.cheyenne_intel.clm-basic
         ERS_Ly3_P72x2.f10_f10_musgs.IHistClm50BgcCropG.cheyenne_intel.clm-cropMonthOutput
         ERS_Ly5_P144x1.f10_f10_musgs.IHistClm50BgcCrop.cheyenne_intel.clm-cropMonthOutput

      I actually expected more tests to show differences, but most tests
      were bit-for-bit. I'm guessing this is because the code added a
      roundoff-level term to a larger term, and this ended up not
      changing the larger term.

      Finally, as an extra precaution, I ran the test suite twice, on
      two different implementations of the truncation: the final one
      (which uses truncate_small_values), and one in which I did the
      truncation using custom, inline code (commit 0d21ccb5). These two
      were bit-for-bit with each other, giving me more confidence that I
      didn't make a mistake with either implementation.


   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev059
Originator(s):  sacks (Bill Sacks)
Date: Sat Aug 24 15:14:06 MDT 2019
One-line Summary: Continue adding water tracers to LakeHydrology

Purpose of changes
------------------

Finish implementing water tracers for initial snow-related code in
LakeHydrology. This involved refactoring code in order to reuse existing
code in SnowHydrology rather than having near-duplicates of that code in
LakeHydrology.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#775 (Implement water tracers for initial
  snow-related code in LakeHydrology)

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none
Many tests took longer, including the PFS test. However, by looking at
more detailed timing numbers in the PFS test, the main increases in time
were in initialization and atm run: lnd run only showed a very small
increase. This suggests that glade slowness was probably primarily
responsible for the increased time.

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: Increased allowed wallclock time for a few tests
These tests took longer, likely due to machine variability. Increasing
their wallclock time to prevent them from timing out.

Code reviewed by: self


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- pass
    hobart ------ pass

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: NO

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev058
Originator(s):  slevis (Samuel Levis,SLevis Consulting LLC,303-665-1310)
Date:  Thu Aug 22 10:02:49 MDT 2019
One-line Summary: Soil texture interpolation bug-fix

Purpose of changes
------------------

 When assigning soil texture values from dataset soil levels to model
 soil levels, we should be comparing zsoi (NOT zisoi) in the model with
 zisoi on the file, i.e. we should be asking whether the node
 center (NOT interface) of a given model level falls between two
 interface depths on the file.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): #772


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[X] clm5_0

[X] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing:
 Unexpected failure of test
 SMS_D_Ld10.f10_f10_musgs.I2000Clm50BgcCropGs.hobart_nag.clm-tracer_consistency
 Bill Sacks did some follow-up testing and concluded that this may be a
 compiler-specific issue, so he changed the test from "nag" to "intel"
 and generated the new test's corresponding baseline

Code reviewed by: @billsacks


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - 

  tools-tests (test/tools):

    cheyenne - 

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - 

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - 

  regular tests (aux_clm):

    cheyenne ---- OK
    hobart ------ OK

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline:

  Summarize any changes to answers, i.e.,
    - what code configurations: not clm45
    - what platforms/compilers: all
    - nature of change: larger than roundoff/same climate
      based on the assumption that shifting soil texture assignments by
      one level in model soil levels below the top-most level should not
      generate climate-changing differences.

Detailed list of changes
------------------------
 1. Changed zisoi to zsoi as described in the Purpose section above.
    This caused the larger than roundoff changes. Before the fix some
    model soil levels were getting assigned soil textures from a deeper
    soil level in the dataset than they should.
 2. In the same section of code I completed the if-block to address all
    zsoi vs. zisoifl comparisons. This ensures explicit assignment of
    soil textures for all model soil levels and avoids leaving some
    model soil levels with the texture assignment of the previous model
    soil level.

Pull Requests that document the changes (include PR ids):
(https://github.com/ESCOMP/ctsm/pull/788)

===============================================================
===============================================================
Tag name:  ctsm1.0.dev057
Originator(s):  sacks (Bill Sacks)
Date: Tue Aug 20 13:17:39 MDT 2019
One-line Summary: Fix frac_sno bugs

Purpose of changes
------------------

Two bug fixes related to frac_sno, and a third change involving
replacing a frac_sno calculation with a simpler equation that is
algebraically equivalent:

(1) Fix lake frac_sno always being 0
    (https://github.com/ESCOMP/ctsm/issues/783). This fixes the albedo
    calculation (and possibly others) over snow-covered lake surfaces.

(2) Fix threshold for explicit snow pack initiation to use frac_sno_eff,
    not frac_sno (https://github.com/ESCOMP/ctsm/issues/785). For
    standard runs (which have use_subgrid_fluxes = .true.), this just
    changes answers for urban columns. This also changes answers more
    widely for runs with use_subgrid_fluxes = .false.

(3) Rewrite Swenson & Lawrence 2012 frac_sno equation to be more
    straightforward and less sensitive to roundoff errors
    (https://github.com/ESCOMP/ctsm/issues/784)

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#783 (frac_sno is always 0 for lake points)
- Resolves ESCOMP/ctsm#785 (Threshold for explicit snow pack initiation
  should use frac_sno_eff, not frac_sno)
- Resolves ESCOMP/ctsm#784 (Suggested algebraic rework of frac_sno
  calculation)

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[X] clm5_0

[X] ctsm5_0-nwp

[X] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: self


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    hobart ------ ok

    ok means tests pass, answers change as expected

    Ran most testing on 95dad328 (before 8f1263a7, which removes a
    temporary endrun error check). Just reran
    SMS_Ld5_D_P48x1.f10_f10_musgs.IHistClm50Bgc.hobart_nag.clm-decStart
    on the final version.

    There was a memleak for this test:

        FAIL SMS_D_Ld10.f10_f10_musgs.I2000Clm50BgcCropGs.hobart_nag.clm-tracer_consistency MEMLEAK memleak detected, memory went from 1250.830000 to 1518.500000 in 8 days

    I have had memleak issues for this test before
    (https://github.com/ESCOMP/ctsm/issues/763), which I have chalked up
    to a compiler-specific issue. I have simply increased the tolerance
    for memleaks for this test moving forward.

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: all
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      Larger than roundoff; expected to be same climate in general,
      because the only impacts are on urban and lake points, which
      represent a small fraction of most grid cells. However, this needs
      further investigation.

      See above (under "Purpose of changes") for a description of the
      different answer changes in this tag.

   If bitwise differences were observed, how did you show they were no worse
   than roundoff?

      I verified that change (3) is no greater than roundoff by
      temporarily putting in an endrun if the new frac_sno differs from
      the old by more than 1e-13; this was not triggered for any test in
      the aux_clm test suite.

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A (Keith Oleson will run this)

   URL for LMWG diagnostics output used to validate new climate: N/A (Keith Oleson will run this)
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev056
Originator(s):  sacks (Bill Sacks)
Date: Fri Aug 16 11:44:43 MDT 2019
One-line Summary: Start adding water tracers to LakeHydrology, and related refactoring

Purpose of changes
------------------

Start adding water tracers to LakeHydrology, beginning with some initial
things done for snow.

This also includes some significant refactoring to allow LakeHydrology
to reuse some of the same snow code used for non-lake columns.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Partially addresses ESCOMP/ctsm#775 (Implement water tracers for
  initial snow-related code in LakeHydrology)

Known bugs found since the previous tag (include github issue ID):
The following will be fixed in an upcoming tag:
- ESCOMP/ctsm#783 (frac_sno is always 0 for lake points)
- ESCOMP/ctsm#784 (Suggested algebraic rework of frac_sno calculation)
- ESCOMP/ctsm#785 (Threshold for explicit snow pack initiation should
  use frac_sno_eff, not frac_sno)

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: self


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    hobart ------ ok

    ok means tests pass, answers change as expected

    Note: Did most testing on 054dc95b (before the small change in
    7e4e52a9); just ran
    LWISO_Ld10.f10_f10_musgs.I2000Clm50BgcCropGs.cheyenne_gnu.clm-coldStart
    on the final commit

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: all
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): roundoff

    There are two answer-changes here:

    (1) Roundoff-level changes due to changing some order of operations
        for the updating of snow_depth and dz for lakes: The previous
        code effectively used (qflx_snow_grnd(c)/bifall(c))*dtime,
        whereas the new code uses (qflx_snow_grnd(c)*dtime)/bifall(c);
        similarly, the dz update differs.

    (2) A change in FSNO_EFF for lakes: previously, this was set to 0
        when there was no snow. Now this is set consistently with other
        landunits, for which it is 1 even if there is no snow. Note that
        this only affects the FSNO_EFF diagnostic field, and nothing
        else.

   If bitwise differences were observed, how did you show they were no worse
   than roundoff?

        I confirmed that the order-of-operations change is the only
        answer-changing part of this tag by comparing against a one-off
        from master with the following diffs; this comparison was
        bit-for-bit except for changes in the FSNO_EFF diagnostic field:

            diff --git a/src/biogeophys/LakeHydrologyMod.F90 b/src/biogeophys/LakeHydrologyMod.F90
            index 91ddfc79..9711dad7 100644
            --- a/src/biogeophys/LakeHydrologyMod.F90
            +++ b/src/biogeophys/LakeHydrologyMod.F90
            @@ -126,6 +126,8 @@ subroutine LakeHydrology(bounds, &
                 real(r8) :: heatsum(bounds%begc:bounds%endc)                ! used in case above [J/m^2]
                 real(r8) :: snowmass                                        ! liquid+ice snow mass in a layer [kg/m2]
                 real(r8) :: snowcap_scl_fct                                 ! temporary factor used to correct for snow capping
            +    real(r8) :: temp_snow_depth
            +    real(r8) :: newsnow
                 real(r8), parameter :: snow_bd = 250._r8                    ! assumed snow bulk density (for lakes w/out resolved snow layers) [kg/m^3]
                                                                             ! Should only be used for frost below.
                 !-----------------------------------------------------------------------
            @@ -246,8 +248,10 @@ subroutine LakeHydrology(bounds, &
                      ! U.S.Department of Agriculture Forest Service, Project F,
                      ! Progress Rep. 1, Alta Avalanche Study Center:Snow Layer Densification.

            -         dz_snowf = qflx_snow_grnd(c)/bifall(c)
            -         snow_depth(c) = snow_depth(c) + dz_snowf*dtime
            +         temp_snow_depth = snow_depth(c)
            +         newsnow = qflx_snow_grnd(c)*dtime
            +         snow_depth(c) = snow_depth(c) + newsnow/bifall(c)
            +         dz_snowf = (snow_depth(c) - temp_snow_depth)/dtime
                      if (snl(c) == 0) then
                         h2osno_no_layers(c) = h2osno_no_layers(c) + qflx_snow_grnd(c)*dtime  ! snow water equivalent (mm)
                      else

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev055
Originator(s):  sacks (Bill Sacks)
Date: Tue Aug  6 14:06:50 MDT 2019
One-line Summary: Modularize snow cover fraction method

Purpose of changes
------------------

This tag moves the calculation of frac_sno - and the related updates of
snow_depth - into a new set of classes, with one class for each
parameterization (Niu & Yang 2007 and Swenson & Lawrence 2012).

Previously, the code always calculated frac_sno the new way, but then
possibly overwrote it if using the older Niu & Yang method. The new code
cleans this up, only doing the calculations that are needed for each
method.

In addition, other code that is specific to one of the two methods is
now moved to a home that makes this dependence on method explicit. This
includes the addition of newsnow to int_snow: previously, int_snow was
always updated using an equation specific to the newer CLM5
parameterization of frac_sno, which was not appropriate if using the Niu
& Yang parameterization; this doesn't make a difference currently, since
int_snow is only referenced if using the Swenson & Lawrence
parameterization, but this clears up some confusion. Also, time-constant
parameters read from namelist or the netCDF parameter file now reside in
the appropriate class rather than being more global.

This tag also renames two namelist options to increase clarity:
- subgridflag is renamed to use_subgrid_fluxes, and is now a logical
- oldfflag is renamed to snow_cover_fraction_method, and is now a string

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#502 (oldfflag can NOT be used with
  subgridflag==1, and rename subgridflag)
- Resolves ESCOMP/ctsm#503 (Clean up CanopyHydrology)
  - This tag does the last piece of cleanup called for in that issue
- Resolves ESCOMP/ctsm#571 (Add a system test: turning on water isotopes
  shouldn't change answers)
  - This is unrelated to the other changes in this tag

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables):
This tag renames two namelist options to increase clarity:
- subgridflag is renamed to use_subgrid_fluxes, and is now a logical
- oldfflag is renamed to snow_cover_fraction_method, and is now a string

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing:
- Added a LWISO test, which confirms that turning on water isotopes
  doesn't change answers for bulk
- Added a test with the newer snow cover fraction method but
  use_subgrid_fluxes false (testmod no_subgrid_fluxes)

Code reviewed by: Sean Swenson, Erik Kluzek


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - ok

    Tests pass; baseline comparisons not done (baseline comparisons
    expected to differ)

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- pass
    hobart ------ pass

  Additional tests: Compared both of these against ctsm1.0.dev052 (which
  should be bit-for-bit with ctsm1.0.dev054): Both were bit-for-bit:

  ERP_D_P36x2_Ld3.f10_f10_musgs.I2000Clm45BgcCrop.cheyenne_gnu.clm-no_subgrid_fluxes
    This is a new test, so did not have baselines; I created baselines
    from ctsm1.0.dev052 and compared against those
  SMS_Ly3.f10_f10_musgs.I2000Clm45BgcCrop.cheyenne_intel.clm-oldhyd_monthly
    This was a temporary test for the sake of running a longer test with
    the oldhyd testmod. This testmod was created as a one-off by
    changing the current oldhyd testmod to inherit from monthly rather
    than default. I ran baselines like this from ctsm1.0.dev052 then ran
    this test from this branch and compared against those baselines.

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: NO

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
https://github.com/ESCOMP/ctsm/pull/769

===============================================================
===============================================================
Tag name:  ctsm1.0.dev054
Originator(s):  sacks (Bill Sacks)
Date: Fri Aug  2 07:57:51 MDT 2019
One-line Summary: Fix interpolation of surfdat soil layers so we can use interpolation for 10SL case

Purpose of changes
------------------

Previously, on master, there was special-purpose code for the 10SL case
that avoided doing interpolation from the surface dataset to the soil
variables in the model. It would be cleaner - especially now that we
allow user-defined soil layer structures - if we could use the general
interpolation code always, rather than sometimes having special-purpose
code that avoids doing the interpolation.

This tag accomplishes that generality. In order to preserve answers for
clm45 cases, I needed to make three changes:

1. The constant 0.025 needed an _r8 appended to it; otherwise, zisoifl
   could differ by roundoff from zisoi.

2. I changed which level is used when zisoi(lev) is exactly equal to
   zisoifl(j) for some j: I changed the conditional in the following:

      if (zisoi(lev) >= zisoifl(j) .AND. zisoi(lev) < zisoifl(j+1)) then
         clay = clay3d(g,j+1)
         sand = sand3d(g,j+1)
         om_frac = organic3d(g,j+1)/organic_max
      endif

   to:

      if (zisoi(lev) > zisoifl(j) .AND. zisoi(lev) <= zisoifl(j+1)) then

   Previously, when the zisoi values in the model exactly lined up with
   the zisoi values in the file, we would set clay in model level j
   equal to the value from level j+1 from the surface dataset (and
   similarly for sand and om_frac); in the new code, we use level j from
   the surface dataset for model level j in this case.

3. I changed the way zisoifl is calculated for the lowest layer, so that
   it matches zisoi(nlevsoi) when running with 10SL_3.5m. Previously, we
   had:

      zisoi(10)   =     0.38018819123227207690E+01
      zisoifl(10) =     0.34330930154359391437E+01

   This tag changes zisoifl(10) to match zisoi(10).

Bugs fixed or introduced
------------------------

Known bugs found since the previous tag (include github issue ID):
- #772: Interpolation of clay, sand and om_frac can use the wrong layer
  (points out more fundamental issues with this interpolation of layers
  from the surface dataset)


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: Sam Levis


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- pass
    hobart ------ pass

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: NO (but see note below)

   In principle, the changes in this tag could change answers for some
   soil layer structures, but there were no answer changes for any of
   the cases covered by the test suite.

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
https://github.com/ESCOMP/ctsm/pull/771

===============================================================
===============================================================
Tag name:  ctsm1.0.dev053
Originator(s):  slevis (Samuel Levis,SLevis Consulting LLC,303-665-1310)
Date: Thu Aug  1 16:56:09 MDT 2019
One-line Summary: Soil layer definition clean-up and user-defined option

Purpose of changes
------------------

 Code clean-up clarifes that there are two types of soil layer
 definition: the node-based and the thickness-based.

 User-defined option allows user to specify a soil layer profile in the
 form of a dzsoi vector (values in meters) in the thickness-based
 approach.

 Default nlevsoi for NWP configurations had to change from 5 to 4 for
 consistency with the new error check described in known bugs below.

 Other code clean-up removes a couple of sections of repeating code.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): #279 #728

Known bugs found since the previous tag (include github issue ID):
 #759 (this PR) bug causes model to abort when nlevsoi = nlevgrnd;
 bug has been corrected with an error check


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[X] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions):
 if neither soil_layerstruct_predefined nor soil_layerstruct_userdefined
 get specified in the namelist, then the model sets
 soil_layerstruct_predefined to the old default setting for
 soil_layerstruct (clm5: 20SL_8.5m, clm4.5: 10SL_3.5m)

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables):
 renamed soil_layerstruct to soil_layerstruct_predefined and added
 soil_layerstruct_userdefined

Changes made to namelist defaults (e.g., changed parameter values):
 Default nlevsoi for NWP configurations had to change from 5 to 4 for
 consistency with the new error check described in known bugs below

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers
--------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing:
 1) New test...
 ERP_P36x2_D_Ld5.f10_f10_musgs.I2000Ctsm50NwpBgcCropGswpGs.cheyenne_intel.clm-default
 replaces existing test
 ERS_D_Ld10.f10_f10_musgs.I2000Clm50BgcCropGs.cheyenne_intel.clm-rm_indiv_lunits_and_collapse_to_dom
 to check the correction described in known bugs above; the new test
 together with existing unit tests cover what the old test was testing

 2) New test and new test type...
 SOILSTRUCTUD_Ld5.f10_f10_musgs.I2000Clm50BgcCropGs.cheyenne_intel.clm-default
 ensures that soil_layerstruct_userdefined gives bfb same answers as
 soil_layerstruct_predefined = '4SL_2m' when set with the same dzsoi
 values. The new test type was put together by:
 - listing the test in (1) testlist_clm.xml (as all tests) and (2) config_tests.xml
 - creating the file .../cime_config/SystemTests/soilstructud.py named after the test in lower case

Code reviewed by: @billsacks


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - 

  tools-tests (test/tools):

    cheyenne - 

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - 

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - 

  regular tests (aux_clm):

    cheyenne ---- OK
    hobart ------ OK

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES (just for NWP configurations)

  Summarize any changes to answers, i.e.,
    - what code configurations: nwp
    - what platforms/compilers: all
    - nature of change: larger than roundoff/same climate
  This is due to the change of nlevsoi from 5 to 4 (more info above).
  I confirmed that nwp does return bfb same answers when I revert this
  change.

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
 https://github.com/ESCOMP/ctsm/pull/759

===============================================================
===============================================================
Tag name:  ctsm1.0.dev052
Originator(s):  sacks (Bill Sacks)
Date: Mon Jul 22 14:02:43 MDT 2019
One-line Summary: Fix rare soil color bug in mksurfdata_map

Purpose of changes
------------------

Under rare conditions, mksurfdata_map could put the default soil color
in an output cell where there is actually more information. This tag
fixes that issue. None of the out-of-the-box surface datasets are
impacted by this bug, so I have not recreated any surface datasets. (I
checked all out-of-the-box surface datasets except for
surfdata_0.125x0.125_mp24_simyr2000_c150114.nc, because it doesn't get
remade cleanly out of the box.)

Also:

- Add some unit tests for the creation of soil color in mksurfdata_map

- Point to correct (existing) surface dataset for year-1850 at f05
  resolution

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#4 (Minor bug in creation of soil color in
  mksurfdata_map: points can be given the default soil color, when they
  should have a real color)
- Resolves ESCOMP/ctsm#765 (Year-1850 f05 surface dataset missing from
  inputdata repository)

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files):
- Year-1850 f05 surface dataset now points to a file that actually exists

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance):

Changes to tests or testing: none

Code reviewed by: self


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - ok

    Most tests pass, including baseline comparisons. The following tests fail, but also failed for me on master:

      019 smiS4 TSMscript_tools.sh ncl_scripts getregional_datasets.pl getregional ....................\c
       rc=6 FAIL
      020 bliS4 TBLscript_tools.sh ncl_scripts getregional_datasets.pl getregional ....................\c
       rc=4 FAIL
      027 smf84 TSMscript_tools.sh PTCLM PTCLMmkdata PTCLM_USUMB_clm4_5^buildtools ....................\c
       rc=6 FAIL
      028 blf84 TBLscript_tools.sh PTCLM PTCLMmkdata PTCLM_USUMB_clm4_5^buildtools ....................\c
       rc=4 FAIL
      029 smfc4 TSMscript_tools.sh PTCLM PTCLMmkdata PTCLM_USUMB_Cycle_clm4_5^buildtools ..............\c
       rc=6 FAIL
      030 blfc4 TBLscript_tools.sh PTCLM PTCLMmkdata PTCLM_USUMB_Cycle_clm4_5^buildtools ..............\c
       rc=4 FAIL
      031 smfg4 TSMscript_tools.sh PTCLM PTCLMmkdata PTCLM_USUMB_Global_clm4_5^buildtools .............\c
       rc=6 FAIL
      032 blfg4 TBLscript_tools.sh PTCLM PTCLMmkdata PTCLM_USUMB_Global_clm4_5^buildtools .............\c
       rc=4 FAIL

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- not run
    hobart ------ not run

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: NO

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev051
Originator(s):  sacks (Bill Sacks)
Date: Fri Jul 19 13:26:25 MDT 2019
One-line Summary: Update water tracers for remainder of first stage of hydrology

Purpose of changes
------------------

Main change is to do tracer updates for final pieces of first stage of
hydrology: HandleNewSnow and UpdateFracH2oSfc. Along with this, I have
introduced greater modularity into these routines (breaking the
operations down into more granular steps).

Other changes included here (somewhat related to the main changes) are:

- Added a routine that resets checked tracers to bulk*ratio to allow
  running the tracer consistency check for more than one time step; we
  now run it for 10 days

- Change logic for rain-snow conversion terms: Use rain diff in rain to
  snow, snow diff in snow to rain. This is less sensitive to roundoff
  errors, and was needed in order to get the 10-day tracer consistency
  test to pass.

- Setting of qflx_snow_h2osfc moved to CanopyInterceptionAndThroughfall

- Add col%lun_itype. col%lun_itype(ci) is the same as
  lun%itype(col%landunit(ci)), but is often a more convenient way to
  access this type. I got tired of having to get the landunit index just
  for the sake of getting the landunit's type of a given column. I have
  NOT gone through the code and replaced lun%itype(l) with
  col%lun_itype(c) where applicable, but this will be useful for the
  future.

- Removed unnecessary code for backwards compatibility of FH2OSFC from
  restart files

- Point to slightly modified initial conditions files for two CLm45
  initial conditions files. These two had FH2OSFC > 0 for some glc_mec
  points. I'm not sure how that came to be, but Sean Swenson and I think
  that shouldn't be the case. This caused soil balance errors in these
  tests:

    ERS_Ly5_P72x1.f10_f10_musgs.IHistClm45BgcCrop.cheyenne_intel.clm-cropMonthOutput
    SMS_D_Ld1.f09_g17.I1850Clm45BgcCruGs.cheyenne_intel.clm-default

  I'm fixing the problem by pointing to updated files with FH2OSFC set
  to 0 for special landunits, using the following python:

      ctypel = dat.variables['cols1d_ityplun'][:]
      dat.variables['FH2OSFC'][ctypel > 2] = 0

- Small cime update to fix the SHAREDLIBBUILD phase of
  ERS_D_Ln9_P480x3.f19_g16.I2000Clm50SpGs.cheyenne_intel.clm-waccmx_offline


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#718 (Implement water tracers for HandleNewSnow
  and FracH2oSfc)
- Resolves ESCOMP/ctsm#498 (Do tracer consistency checks every time
  step, not just first)

CIME Issues fixed (include issue #):
- ESCMI/cime#3717 (Cheyenne ESMF missing for WACCM X)

Known bugs introduced in this tag (include github issue ID):
- ESCOMP/ctsm#762 (Water tracers: Make sure calls to ResetCheckedTracers
  are removed)

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions):
- If you have an initial conditions file with non-zero FH2OSFC for
  glacier (this was the case for some out-of-the-box clm45 files, which
  are now fixed), may run into soil balance errors. To fix this, set
  FH2OSFC to 0 in your restart file for all special landunits.

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files):
- Point to slightly modified initial conditions files for two CLm45
  initial conditions files: see above for details.

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing:
- Changed tracer consistency test to 10 days rather than a single time step

Code reviewed by: self


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    hobart ------ ok

    ok means tests pass, some roundoff-level diffs in baseline
    comparisons, as noted below

  Also, in order to have a more powerful test of the changes in
  SurfaceWaterMod (since the relevant code isn't triggered very often),
  I introduced the following diffs:

    diff --git a/src/biogeophys/SurfaceWaterMod.F90 b/src/biogeophys/SurfaceWaterMod.F90
    index 1bc03cae..be656a15 100644
    --- a/src/biogeophys/SurfaceWaterMod.F90
    +++ b/src/biogeophys/SurfaceWaterMod.F90
    @@ -12,6 +12,7 @@ module SurfaceWaterMod
       use shr_spfn_mod                , only : erf => shr_spfn_erf
       use clm_varcon                  , only : denh2o, denice, roverg, wimp, tfrz, pc, mu, rpi
       use clm_varpar                  , only : nlevsno, nlevgrnd
    +  use clm_varctl                  , only : iulog
       use clm_time_manager            , only : get_step_size
       use column_varcon               , only : icol_roof, icol_road_imperv, icol_sunwall, icol_shadewall, icol_road_perv
       use decompMod                   , only : bounds_type
    @@ -201,7 +202,7 @@ subroutine BulkDiag_FracH2oSfc(bounds, num_soilc, filter_soilc, &
         SHR_ASSERT_FL((ubound(qflx_too_small_h2osfc_to_soil, 1) == bounds%endc), sourcefile, __LINE__)
     
         ! arbitrary lower limit on h2osfc for safer numerics...
    -    min_h2osfc=1.e-8_r8
    +    min_h2osfc=1.e0_r8
     
         do f = 1, num_soilc
            c = filter_soilc(f)
    @@ -230,6 +231,9 @@ subroutine BulkDiag_FracH2oSfc(bounds, num_soilc, filter_soilc, &
     
            else
               frac_h2osfc(c) = 0._r8
    +          if (h2osfc(c) > 0._r8) then
    +             write(iulog,*) 'WJS: in else'
    +          end if
               qflx_too_small_h2osfc_to_soil(c) = h2osfc(c) / dtime
               ! The update of h2osfc is deferred to later, keeping with our standard
               ! separation of flux calculations from state updates, and because the state

  With those diffs, I ran the tracer consistency test; it passed. In
  addition, I ran
  SMS_D_Ld1_P4x1.f10_f10_musgs.I2000Clm50BgcCropQianRsGs.bishorn_gnu.clm-default
  with comparison against master, with the following diffs on master
  (the first change changes behavior to force the relevant code to be
  triggered, and is the same on master and on the branch; the second
  change on master divides then multiplies by dtime to avoid
  roundoff-level diffs):

    diff --git a/src/biogeophys/SurfaceWaterMod.F90 b/src/biogeophys/SurfaceWaterMod.F90
    index 959d539b..4a4a4824 100644
    --- a/src/biogeophys/SurfaceWaterMod.F90
    +++ b/src/biogeophys/SurfaceWaterMod.F90
    @@ -77,7 +77,7 @@ subroutine FracH2oSfc(bounds, num_nolakec, filter_nolakec, &
              )
     
         ! arbitrary lower limit on h2osfc for safer numerics...
    -    min_h2osfc=1.e-8_r8
    +    min_h2osfc=1.e0_r8
     
         call waterstatebulk_inst%CalculateTotalH2osno(bounds, num_nolakec, filter_nolakec, &
              caller = 'FracH2oSfc', &
    @@ -112,7 +112,7 @@ subroutine FracH2oSfc(bounds, num_nolakec, filter_nolakec, &
     
               else
                  frac_h2osfc(c) = 0._r8
    -             h2osoi_liq(c,1) = h2osoi_liq(c,1) + h2osfc(c)
    +             h2osoi_liq(c,1) = h2osoi_liq(c,1) + (h2osfc(c) / get_step_size()) * get_step_size()
                  h2osfc(c)=0._r8
               endif

  This test passed and was bit-for-bit with master with the above changes.


If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline:

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: Many/all
    - what platforms/compilers: All
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      Roundoff: just roundoff-level changes in FSH_PRECIP_CONVERSION,
      FSH_TO_COUPLER, l2x_Fall_sen due to refactored calculation of
      FSH_PRECIP_CONVERSION

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? via summarize_cprnc_diffs

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.):
- cime: branch_tags/cime5.8.3_chint17-03 -> branch_tags/cime5.8.3_chint17-04
  Fixes ESMCI/cime#3171

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev050
Originator(s):  slevis (Samuel Levis,SLevisConsulting LLC,303-665-1310)
Date:  Mon Jul 15 11:40:09 MDT 2019
One-line Summary: dz --> dz_lake bug-fix in LakeTemperatureMod.F90 line 960

Purpose of changes
------------------

 Bug-fix to prevent the model from aborting when running with fewer soil
 layers than lake layers; not to imply that this was not a bug when the
 model wasn't aborting. It was.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): #760

Known bugs found since the previous tag (include github issue ID): #759 bug causing model to abort when nlevsoi = nlevgrnd


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers
--------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: @dlawrenncar @billsacks


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - 

  tools-tests (test/tools):

    cheyenne - 

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - 

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - 

  regular tests (aux_clm):

    cheyenne ---- OK
    hobart ------ OK

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline:

  Summarize any changes to answers, i.e.,
    - what code configurations: use_lch4 = .true.
    - what platforms/compilers: all
    - nature of change: diagnostic variable WTGQ only
                 Confirmed this on cheynne and hobart by running
                 ./summarize_cprnc_diffs -baseline .../tests_0712... -testid '*'
                 and inspecting the file cprnc.summary.*.by_varname

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
 https://github.com/ESCOMP/ctsm/pull/761

===============================================================
===============================================================
Tag name:  ctsm1.0.dev049
Originator(s):  erik (Erik Kluzek)
Date: Sun Jun 23 20:56:55 MDT 2019
One-line Summary: Update mosart and intel to intel-19 on cheyenne

Purpose of changes
------------------

Update mosart to trunk tag (that has a roundoff change to history file
output). And update intel compiler on cheyenne to intel-19.

Bugs fixed or introduced None
------------------------

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): None

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): None

Changes made to namelist defaults (e.g., changed parameter values): None

Changes to the datasets (e.g., parameter, surface or initial files): None

Substantial timing or memory changes: [For timing changes, can check PFS test(s) in the test suite] None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): None

Changes to tests or testing: None

Code reviewed by: self

CTSM testing: regular

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS

  regular tests (aux_clm):

    cheyenne ---- OK
    hobart ------ OK

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: Yes

  Summarize any changes to answers, i.e.,
    - what code configurations: some compsets with mosart
    - what platforms/compilers: cheyenne/intel
    - nature of change: roundoff

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): cime, and mosart
    mosart to mosart1_0_33
    cime   to branch_tags/cime5.8.3_chint17-03

Pull Requests that document the changes (include PR ids): #753
(https://github.com/ESCOMP/ctsm/pull)
   #753 -- Update cheyenne to intel/19 and update mosart

===============================================================
===============================================================
Tag name:  ctsm1.0.dev048
Originator(s):  erik (Erik Kluzek,UCAR/TSS,303-497-1326)
Date: Sun Jun 23 15:16:01 MDT 2019
One-line Summary: Updates for buildlib changes and cime and externals updates

Purpose of changes
------------------

Update of most externals: cime, mosart, rtm, cism. The cime update also includes a required CESM-wide
change in buildlib. Uses a function to get the Macros filename "get_standard_makefile_args".
The cime update brought in some answer chagnes, with an update of the intel compiler (but we backed those
out by using a cime branch). There is also a roundoff change by going to the trunk version of RTM.

Bugs fixed or introduced None
------------------------

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): None

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): None

Changes made to namelist defaults (e.g., changed parameter values): None

Changes to the datasets (e.g., parameter, surface or initial files): None

Substantial timing or memory changes: None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): None

Changes to tests or testing: None

Code reviewed by: @jedwards, self


CTSM testing: regular

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS

  tools-tests (test/tools):

    cheyenne - PASS

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - PASS

  python testing (see instructions in python/README.md; document testing done):

     cheyenne - PASS (for python3)

  regular tests (aux_clm):

    cheyenne ---- OK
    hobart ------ OK

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: Yes, but only for a few cases (compsets with RTM)

  Summarize any changes to answers, i.e.,
    - what code configurations: With RTM
    - what platforms/compilers: All
    - nature of change: roundoff
       fieldlist is different for CISM in some cases

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): cime, mosart, rtm, cism
    cime   to branch_tags/cime5.8.3_chint17-02
    cism   to cism2_0_68
    rtm    to rtm1_0_68
    mosart to nldas-grid.n02_mosart1_0_31

Pull Requests that document the changes (include PR ids): #752
(https://github.com/ESCOMP/ctsm/pull)

   #752 -- buildlib, cime and externals update

===============================================================
===============================================================
Tag name:  ctsm1.0.dev047
Originator(s):  sacks (Bill Sacks)
Date:  Sun Jun 16 13:04:38 MDT 2019
One-line Summary: Fix negative snow compaction during snow melt

Purpose of changes
------------------

Fix an issue reported by @kjetilaas: In the presence of surface water,
the old and new snow cover fractions are inconsistently calculated in
subroutine "SnowCompaction". This can result in significant negative
compaction (snow depth increase) during snow melt.

More details from @kjetilaas (copied from
https://github.com/escomp/ctsm/issues/573):

There is an inconsistency in the calculation of snow cover fraction
(FSNO) in the SnowCompaction subroutine when there is surface water
present. The actual snow cover is limited to 0.99 when surface water is
present (and the surface water fraction is set to 0.01). However,
SnowCompaction recalculates the old FSNO without accounting for this
limitation. This results in an apparent decrease in FSNO, and
corresponding negative snow compaction from snow melt (ddz3).

This can result in substantial, artificial increase in snow depth during
summer (can be more than 0.5 m during a single season).

The problem might be masked in monthly output, but could be significant
whenever snow cover is close to 1 and surface water present. Snow mass
is indirectly affected by this as radiation and temperature fluxes in
the snow pack will be affected by artificially thick snow layers.

This tag fixes this issue. The fix was implemented and tested by
@swensosc (Sean Swenson).

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#573 (Snow depth increase due to inconsistent snow
  cover fraction calculation)


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[X] clm5_0

[X] ctsm5_0-nwp

[X] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: Sean Swenson, @kjetilaas, Bill Sacks


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    hobart ------ ok

    ok means tests pass, answers change as expected

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: all
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate):
      Larger than roundoff/same climate

      Sean Swenson ran the LMWG diagnostics package and looked at the
      results with Keith Oleson. Their main findings were:

      - Snow depth is a bit smaller over Greenland and polar regions on
        average, as expected (up to 5-10% decrease in average snow depth
        averaged over the Canadian Arctic)

      - There is a small increase in h2osno over Greenland: about 30 mm
        averaged over Greenland; it looks like the main changes are
        around the coasts (which makes sense given that this bug only
        affects vegetated landunits)

      - There are small changes in runoff over Greenland

      - Changes in snow cover fraction are tiny

      - Overall, this doesn't look climate changing, but could
        potentially have a significant impact on select grid cells, and
        on some variables in polar regions

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev046
Originator(s):  sacks (Bill Sacks)
Date:  Sat Jun 15 15:54:44 MDT 2019
One-line Summary: Separate the two uses of h2osno

Purpose of changes
------------------

Until now, h2osno has been used in two ways:

1. When there are explicit snow layers, h2osno is the sum of snow
   (liquid + ice) in all layers

2. When there is a little bit of snow, but not enough to create a snow
   layer, h2osno gives the amount of this very thin snow pack (assumed
   to be all ice)

This is confusing and complicates the addition of water tracers to the
code. This tag separates h2osno into two different variables:

1. h2osno_no_layers is a fundamental state variable that gives the
   amount of snow in the case where we don't have explicit snow layers

2. h2osno_total is purely a diagnostic variable that is the sum of snow
   in all explicit layers, plus h2osno_unresolved. This exists in
   waterdiagnosticbulk_type, but the version there is just set once at
   the end of the time step, so that version is only meant for
   diagnostic (history) output. Code that needs the current sum of
   h2osno at a given point now computes it locally, via a new method on
   waterstate_type: CalculateTotalH2osno. If running in debug mode, that
   routine also performs some consistency checks.

Also, an unrelated change to prevent user from trying to interpolate a
file onto itself.

Also, add a compset for testing, which is good for use on my mac.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#733 (Make h2osno purely a diagnostic variable)
- Resolves ESCOMP/ctsm#749 (h2osno inconsistent when interpolating from
  initial conditions with different snow layers)
- Resolves ESCOMP/ctsm#750 (Prevent user from trying to use the same
  filename for init_interp source and destination files)

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none (possibly a small improvement
in timing based on PFS run time)

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: self


CTSM testing:


 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    hobart ------ ok

    ok means tests pass; baseline comparisons fail as expected

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline:

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: all
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      Mostly roundoff; greater than roundoff/same climate for cases for
      which number of snow layers is less in the case than in its
      initial conditions file, due to fixing #749

   If bitwise differences were observed, how did you show they were no worse
   than roundoff?

   There was one point where h2osno_total differed from the old h2osno
   by roundoff, as verified by a run of the test suite (where I verified
   that (a) it only differed by 1e-11 or less, and (b) if I reset it,
   there were just roundoff-level changes in a few diagnostic
   fields). That one place with roundoff-level diffs can lead to
   propagation of diffs, causing differences in many fields in the final
   run of the test suite. In particular: I ran the test suite on
   69823de4 (which had this temporary check and reset of h2osno_total
   vs. old h2osno). For nearly all tests, there were only roundoff-level
   changes in a few diagnostic fields. I then removed the temporary
   check and reset and ran the test suite on the result
   (4353cc90). Finally, I ran the test suite on the final, cleaned-up
   code, with comparison against 4353cc90 and verified bfb.

   In my testing on 69823de4: There were a few tests with greater than
   roundoff-level changes due just to differences in the initial value
   of h2osno_total (which was incorrect in the old due to #749). I
   verified that these only had roundoff-level changes in a few
   diagnostic fields if I used the following diffs in both the baseline
   and my branch, indicating that the diffs just arose due to the fix of
   #749:

        diff --git a/src/main/clm_initializeMod.F90 b/src/main/clm_initializeMod.F90
        index 793b36ac..f10ee5e1 100644
        --- a/src/main/clm_initializeMod.F90
        +++ b/src/main/clm_initializeMod.F90
        @@ -762,6 +762,18 @@ subroutine initialize2( )
                !$OMP END PARALLEL DO
             end if
        +    do c = bounds_proc%begc, bounds_proc%endc
        +       if (col%snl(c) < 0) then
        +          water_inst%waterstatebulk_inst%h2osno_col(c) = 0._r8
        +          do j = col%snl(c)+1, 0
        +             water_inst%waterstatebulk_inst%h2osno_col(c) = &
        +                  water_inst%waterstatebulk_inst%h2osno_col(c) + &
        +                  water_inst%waterstatebulk_inst%h2osoi_ice_col(c,j) + &
        +                  water_inst%waterstatebulk_inst%h2osoi_liq_col(c,j)
        +          end do
        +       end if
        +    end do
        +
           end subroutine initialize2
         end module clm_initializeMod


   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: n/a

   URL for LMWG diagnostics output used to validate new climate: n/a
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev045
Originator(s):  sacks (Bill Sacks)
Date: Thu Jun  6 12:26:57 MDT 2019
One-line Summary: Recalculate h2osno for the sake of SnowCapping

Purpose of changes
------------------

SnowCapping decides whether to remove excess snow based on
h2osno. However, at the point where this is called,h2osno is out of sync
with (h2osoi_liq + h2osoi_ice): Those variables can be changed slightly
in SnowWater (which is called before SnowCapping), yet h2osno isn't
recalculated based on those changes.

I don't think this causes a big problem in practice, but it seems like
it would improve code understandability if we had the general rule that
any code that references h2osno is using a version of h2osno that is in
sync with (h2osoi_liq + h2osoi_ice). This is also needed to prevent
answer changes in an upcoming major refactor.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#736

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: Sean Swenson signed off on the change, but didn't
review the final code


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    hobart ------ ok

    ok means tests pass, answers change as expected

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: all
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      Larger than roundoff/same climate

    Answer changes are expected to be small, since this should just
    change the timing of snow capping a bit, but I haven't looked
    closely at the magnitude of the answer changes.


   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev044
Originator(s):  sacks (Bill Sacks)
Date: Thu Jun  6 12:14:39 MDT 2019
One-line Summary: Make wetland snow resetting behavior more explicit

Purpose of changes
------------------

There is some code that zeroes out h2osno over thawed wetlands. It seems
like the intended behavior here is to zero out the entire snow
pack. Currently, however, this only zeros out a very thin (zero-layer)
snow pack. For now, I'm adding an snl==0 conditional to make this
behavior explicit; long-term, we'd like to change this to actually zero
out the whole snow pack. (At that time, this code block should probably
be moved to a more appropriate home, as noted in comments in issue
ESCOMP/ctsm#735.)

Also, unrelated change in run_sys_tests: support machine-specific
baseline directory, in order to support new default location for CTSM
baselines on cheyenne.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Partially addresses, but does NOT fix, ESCOMP/ctsm#735

Known bugs introduced in this tag (include github issue ID):
- Partially addresses, but does NOT fix, ESCOMP/ctsm#735


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: Sean Swenson and Keith Oleson approved this change,
though didn't actually review the final code.


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

    cheyenne - not run

  python testing (see instructions in python/README.md; document testing done):

    (any machine) - pass on cgdm-bishorn
    make python=python2 test
    make pyhton=python3 test
    make lint

  regular tests (aux_clm):

    cheyenne ---- pass
    hobart ------ pass

    Also ran SMS_Ly1.f09_g17.I2000Clm50SpGs.cheyenne_intel.clm-monthly
    with comparison against baselines: passed and bit-for-bit.

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: NO (but see notes below)

   It seems theoretically possible for this to change answers, but no
   answer changes were observed in the aux_clm test suite or
   SMS_Ly1.f09_g17.I2000Clm50SpGs.cheyenne_intel.clm-monthly.


Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev043
Originator(s):  erik (Erik Kluzek)
Date: Wed Jun  5 10:17:06 MDT 2019
One-line Summary: Fix FUN bug (frac_ideal_C_use was backwards in regard to delta_CN), and replace Ball-Berry mbbopt with the CLM4.5 version

Purpose of changes
------------------

Carbon allocation to uptake responds to CN(uptake-cost) and CN(actual). The intended implementation is: For C:N 
less than the target C:N, C allocation is reduced with cost. For C:N greater than the target C:N, C allocation 
is increased with high C:N. However, the actual implementation is reversed. This change fixes that problem.

Also the Ball-Berry mbbopt values had been changed in the creation of CLM5.0, as part of the tuning, but the new
values don't work well (as our final tuning uses Medlyn photosynshesis rather than Ball-Berry). This brings the values
for Ball-Berry for CLM5.0 back to the CLM4.5 values.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):  #704 #705

   Fixes #705 -- Ball-Berry parameters in CLM5.0 should go back to the previous values in CLM4.5
   Fixes #704 -- FUN code logic to reduce or increase carbon allocation used for uptake is reversed

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[X] clm5_0

[X] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): None

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): None

Changes made to namelist defaults (e.g., changed parameter values): New clm5.0 paramdata file

Changes to the datasets (e.g., parameter, surface or initial files): mbbopt now use the clm4_5 values
   rather than the pretuned clm5_0 values (before Medlyn was being used)

Substantial timing or memory changes: None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): None

Changes to tests or testing: None

Code reviewed by: self, olyson


CTSM testing: regular

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS (134 tests are different from baseline as expected)

  regular tests (aux_clm):

    cheyenne ---- OK
    hobart ------ OK

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: Yes!

  Summarize any changes to answers, i.e.,
    - what code configurations: Anything with use_FUN=T (so Clm50Bgc compsets) 
        Also with Clm5.0 and Ball-Berry (stomatalcond_method='Ball-Berry1987') (so Clm50Nwp compsets)
    - what platforms/compilers: All
    - nature of change: climate is similar

       "Our assessment is that the impact is relatively small, but would affect climate; 
        however it doesn’t appear to strongly affect transient C response."

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
      Experiments were done on the release branch
       - casename: oleson/clm50_release-clm5.0.15_delta_CN_FUNbug_2deg_GSWP3V1_1850
                   oleson/clm50_release-clm5.0.15_delta_CN_FUNbug_2deg_GSWP3V1_hist

   URL for LMWG diagnostics output used to validate new climate:

http://webext.cgd.ucar.edu/I1850/clm50_release-clm5.0.15_delta_CN_FUNbug_2deg_GSWP3V1_1850/lnd/clm50_release-clm5.0.15_delta_CN_FUNbug_2deg_GSWP3V1_1850.21_40-clm50_release-clm5.0.15_2deg_GSWP3V1_1850.21_40/setsIndex.html

We've also completed a historical run. Diagnostics are here:

http://webext.cgd.ucar.edu/I20TR/clm50_release-clm5.0.15_delta_CN_FUNbug_2deg_GSWP3V1_hist/lnd/clm50_release-clm5.0.15_delta_CN_FUNbug_2deg_GSWP3V1_hist.1995_2014-clm50_release-clm5.0.15_2deg_GSWP3V1_hist.1995_2014/setsIndex.html

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): None

Pull Requests that document the changes (include PR ids): #727
(https://github.com/ESCOMP/ctsm/pull)

  #727 -- Fix the FUN bug, where an if statement was backwards

===============================================================
===============================================================
Tag name:  ctsm1.0.dev042
Originator(s):  slevis (Samuel Levis,SLevis Consulting LLC,303-665-1310)
Date: Tue May 21 23:29:31 MDT 2019
One-line Summary: Rename, correct, and simplify parameters dewmx and sno_stor_max

Purpose of changes
------------------

 To rename
 dewmx to liq_canopy_storage_scalar
 sno_stor_max to sno_canopy_storage_scalar
 in the code and in the clm45 and clm50 param files.

 To change snocanmx, liqcanmx, and fcansno to...
 snocanmx = params_inst%snow_canopy_storage_scalar(p) * (elai(p) + esai(p))
 liqcanmx = params_inst%liq_canopy_storage_scalar(p) * (elai(p) + esai(p))
 fcansno(p) = (snocan(p) / (vegt * params_inst%sno_stor_max))**0.15_r8

 Rewrote fwet(p) to follow the same format as fcansno(p).


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): #710
   Fixes #710 -- rename dewmx


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): None

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): None

Changes made to namelist defaults (e.g., changed parameter values): New default parameter file

Changes to the datasets (e.g., parameter, surface or initial files):
 Pointing to new clm45 and clm50 param files containing the two renamed
 parameters listed above.

Substantial timing or memory changes: None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): None

Changes to tests or testing: None

Code reviewed by:
 @billsacks

CTSM testing: regular

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS (196 different because of new param file)

  regular tests (aux_clm):

    cheyenne ---- OK
    hobart ------ OK
 Roundoff changes were expected.

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: Yes!

  Summarize any changes to answers, i.e.,
    - what code configurations: ALL
    - what platforms/compilers: ALL
    - nature of change: roundoff

   If bitwise differences were observed, how did you show they were no worse
   than roundoff?
   The code modifications causing the change were of the order-of-calculations
   type in two lines of code.

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): None

Pull Requests that document the changes (include PR ids):
 https://github.com/ESCOMP/ctsm/pull/719

===============================================================
===============================================================
Tag name:  ctsm1.0.dev041
Originator(s):  sacks (Bill Sacks)
Date:  Fri May 17 06:02:38 MDT 2019
One-line Summary: Add water tracers to CanopyHydrologyMod

Purpose of changes
------------------

Major overhaul of CanopyHydrologyMod in order to support water tracers.

The part of CanopyHydrology that was actually dealing with canopy
hydrology has now been broken out into a number of small routines in
order to separate flux calculations from state updates. This is needed
in order to calculate tracer versions of the relevant variables, which
is now done here.

The parts of CanopyHydrology that were NOT dealing with canopy hydrology
have been moved to more appropriate homes. They have not yet been
tracerized, but will be soon.

This also makes some minor changes to diagnostic fields related to
canopy hydrology. Of particular note: QDRIP now has a different meaning:
it now corresponds to the excess liquid water that runs off from the
canopy (and is inactive by default).

The design of this broken-out code was developed in consultation with
Mike Barlage, Erik Kluzek, Negin Sobhani, Sean Swenson and Mariana
Vertenstein. We plan to apply this design moving forward to other parts
of the code that need tracerization. See also
<https://github.com/billsacks/prototypes-ctsm-canopyhydrology_tracers>.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#360 - Implement water isotopes for
  CanopyHydrology
- Resolves ESCOMP/ctsm#709 - QDRIP diagnostic field misleadingly named
- Partially addresses ESCOMP/ctsm#503 - Clean up CanopyHydrology
  (We still need to clean up / tracerize the parts that have now been
  moved elsewhere.)


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions):
- Changed meaning of QDRIP diagnostic field: it now corresponds to
  qflx_liqcanfall rather than qflx_prec_grnd_patch, and is default
  inactive. (There is no longer a diagnostic field corresponding to
  qflx_prec_grnd_patch, but there are separate diagnostic fields for the
  rain and snow portions of this, both default inactive.)

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none
- checked timing of hydro1 timer vs. previous canhydro timer from my
  dev039 testing: differences are negligible

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by:
- Design reviewed by Mike Barlage, Erik Kluzek, Negin Sobhani, Sean
  Swenson and Mariana Vertenstein
- Final code reviewed only by self

CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    hobart ------ ok

    ok means tests pass, baseline comparisons fail as expected

If the tag used for baseline comparisons was NOT the previous tag, note that here:


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: all
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      roundoff

   If bitwise differences were observed, how did you show they were no worse
   than roundoff?

     Ensured no-greater-than-roundoff-level by:

     First, generated baselines using code with these diffs (note extra
     parentheses in multiplication):

       diff --git a/src/biogeophys/CanopyHydrologyMod.F90 b/src/biogeophys/CanopyHydrologyMod.F90
       index f848f520..31c8c550 100644
       --- a/src/biogeophys/CanopyHydrologyMod.F90
       +++ b/src/biogeophys/CanopyHydrologyMod.F90
       @@ -338,8 +338,8 @@ subroutine CanopyHydrology(bounds, &
                           ! Intercepted precipitation [mm/s]
                           qflx_prec_intr(p) = forc_snow(c)*fpisnow + qflx_liq_above_canopy(p)*fpi
                           ! storage of intercepted snowfall, rain, and dew
       -                   snocan(p) = max(0._r8, snocan(p) + dtime*forc_snow(c)*fpisnow)
       -                   liqcan(p) = max(0._r8, liqcan(p) + dtime*qflx_liq_above_canopy(p)*fpi)
       +                   snocan(p) = max(0._r8, snocan(p) + dtime*(forc_snow(c)*fpisnow))
       +                   liqcan(p) = max(0._r8, liqcan(p) + dtime*(qflx_liq_above_canopy(p)*fpi))

                           ! Initialize rate of canopy runoff and snow falling off canopy
                           qflx_snocanfall(p) = 0._r8


     Then ran test suite on 1cc9d38a. This is very close to the final
     branch, but had a few small tweaks in place to get bit-for-bit:

     1. Under some conditions, the final branch code resets slightly
        negative snocan and liqcan to 0, where master did not. This
        commit puts in place temporary code to maintain the slightly
        negative values under these conditions, along with a check to
        ensure that the values are really always only slightly negative
        in this situation.

     2. When snocan > snocanmx, master reset snocan to exactly snocanmx,
        whereas the final branch code can result in snocan being
        roundoff-level different from snocanmx, and similar for
        liqcan. This commit puts in place temporary code to set snocan
        to snocanmx under appropriate circumstances, and liqcan to
        liqcanmx, along with a check to ensure that this is only leading
        to roundoff-level changes in these state variables.

     3. Multiplies then divides a flux by dtime in one place

     4. Has parentheses around a partial sum to ensure same order of
        operations as before

     This was bfb with the baselines generated with the diff noted
     above, except for roundoff-level changes in SNOCAN in one grid cell
     in just one test
     (SMS_D_Ld1.f19_g17.I1850Clm45Cn.hobart_gnu.clm-default; old:
     -8.15e-322, new: -2e-323).

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
None. But see
https://github.com/billsacks/prototypes-ctsm-canopyhydrology_tracers,
which was the source of our design discussions and contains notes from
these discussions.

===============================================================
===============================================================
Tag name:  ctsm1.0.dev040
Originator(s):  slevis (Samuel Levis,SLevis Consulting LLC,303-665-1310)
Date:  Fri May 03 10:34:46 MDT 2019
One-line Summary: Move some hard-coded parameters from code to params.nc file

Purpose of changes
------------------

 Reduce the number of parameters that get hardwired directly in the code
 by placing in the params.nc file where they become visible and easier
 to modify.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): #680

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users:
 None

Changes to CTSM's user interface:
 Parameters that were hardwired in the code can now be seen and changed
 in the params.nc file.

Changes made to namelist defaults (e.g., changed parameter values):
 None

Changes to the datasets (e.g., parameter, surface or initial files):
 Various parameters have moved out of the code and into the params.nc file.

Substantial timing or memory changes: [For timing changes, can check PFS test(s) in the test suite]

Notes of particular relevance for developers:
 None
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance):
 None

Changes to tests or testing:
 None

Code reviewed by:
 @ekluzek
 @billsacks

CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - 

  tools-tests (test/tools):

    cheyenne - 

  PTCLM testing (tools/shared/PTCLM/test):

     cheyenne - 

  regular tests (aux_clm):

    cheyenne ---- OK
    hobart ------ OK
 Answers change as expected.

CTSM tag used for the baseline comparisons: 
 ctsm1.0.dev039

Answer changes
--------------

Changes answers relative to baseline: YES
  Summarize any changes to answers, i.e.,
    - what code configurations: ALL
    - what platforms/compilers: ALL
    - nature of change: single-precision roundoff
    Changes are due to four parameters:
 accum_factor 
 ko25_coef 
 kc25_coef 
 frac_sat_soil_dsl_init 
    that were not declared "_r8" when hardwired in the code,
    while now they are declared "double" in the params.nc files. I
    showed that the change was no greater than single-precision roundoff
    by writing the new and old values to the lnd and cesm log files and
    confirming differences smaller than 1.e-6.

Detailed list of changes
------------------------

 - Updated the clm5 and clm4.5 params.nc files to contain the previously
 hard-coded parameters that Katie Dagon recommended moving out of the
 code (listed here: https://docs.google.com/spreadsheets/d/1cRmcB0xz7CjVP6ljUoCehJx49Usxsore5ma4mWcvQwM/edit#gid=0)
 - Now read the parameters in the modules that use them

Pull Requests that document the changes (include PR ids):
 https://github.com/ESCOMP/ctsm/pull/684

===============================================================
===============================================================
Tag name:  ctsm1.0.dev039
Originator(s):  sacks (Bill Sacks)
Date: Wed May  1 11:12:34 MDT 2019
One-line Summary: Remove excess canopy liquid/snow regardless of temperature

Purpose of changes
------------------

Previously: if it was above freezing, snow was allowed to accumulate on
the canopy beyond the specified capacity, and if it was below freezing,
liquid was allowed to accumulate beyond the specified capacity. Sean
Swenson, Keith Oleson and Dave Lawrence agreed that this seems wrong.

This tag changes the behavior so that both liquid and snow canopy
capacities are enforced regardless of temperature.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#699 (Canopy liquid and snow capacities not
  enforced consistently?)


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[X] clm5_0

[X] ctsm5_0-nwp

[X] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: Keith Oleson


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    hobart ------ ok

    ok means tests pass, answers change as expected

CTSM tag used for the baseline comparisons: ctsm1.0.dev038


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: all
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      larger than roundoff; likely same climate, but this still requires
      investigation

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A (Keith Oleson will run this soon)

   URL for LMWG diagnostics output used to validate new climate: N/A
   (Keith Oleson will run this soon)
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
https://github.com/ESCOMP/ctsm/pull/703

===============================================================
===============================================================
Tag name:  ctsm1.0.dev038
Originator(s):  sacks (Bill Sacks)
Date: Wed May  1 10:53:02 MDT 2019
One-line Summary: Support for NWP configuration, NLDAS grid and NLDAS datm forcing

Purpose of changes
------------------

This tag includes a set of changes that, together, provide support for
running an initial Numerical Weather Prediction version of CTSM over the
Continental U.S.

(1) NWP configuration: The changes in this configuration relative to the
    climate (CLM5) configuration mainly target decreasing runtime by
    removing features that are expensive and not as important for NWP
    time scales (days to months rather than many years) and space scales
    (high resolution):

    - Single dominant landunit; if vegetated, single dominant PFT

    - Only 5 soil layers, down to 3 m

    - Only 5 snow layers

    - Plant hydraulic stress off

    - Ball-Berry rather than Medlyn stomatal conductance method (this is
      a side-effect of turning plant hydraulic stress off, and is not
      itself a critical aspect of the NWP configuration)

    - Maximum of 3 iterations to compute canopy fluxes

    - MEGAN is off

    Note that the NWP compset triggers changes in two new xml variables:
    CLM_CONFIGURATION ('clm' vs. 'nwp'; this controls things of a more
    scientific nature, like whether plant hydraulic stress is on or
    off), and CLM_STRUCTURE ('standard' vs. 'fast'; this controls
    structural things like the maximum number of landunits per grid cell
    and the soil layer structure). Thus, you can create a case that is a
    hybrid between the CLM and NWP configurations by changing these two
    xml variables independently.

(2) NLDAS2 regional grid: 0.125 deg grid over the Continental U.S.

(3) Updated version of MOSART with support for the same NLDAS2 regional
    grid

(4) NLDAS2 data atmosphere forcing: over the same regional grid; forcing
    data are available from 1980 - 2018. (We have excluded 1979 because
    a small amount of data were missing for that year, and we didn't
    have a good way to handle those missing data.)

These changes can be used independently or all together. However, note
that you will get garbage if you try to use the new NLDAS2 datm forcing
for a run that extends beyond the boundaries of the NLDAS2 domain.

There are three new NWP compsets:
- I2000Ctsm50NwpSpGswpGs: GSWP3 datm forcing; meant for global runs or
  regional runs outside the NLDAS domain
- I2000Ctsm50NwpSpNldasGs: NLDAS2 datm forcing; meant for regional runs
  over the NLDAS domain or some portion of it
- I2000Ctsm50NwpSpNldasRsGs: Same as above, but with a stub runoff model
  in place of MOSART, for runs that aren't interested in having a runoff
  model and for which you want improved throughput

The alias for the new grid is: nldas2_rnldas2_mnldas2 (indicating that
we're using the nldas2 grid for land/atmosphere, runoff ('r') and the
ocean mask ('m').


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#451 (Add NWP physics option)
- Resolves ESCOMP/ctsm#452 (Add an NWP compset)


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions):
- The NWP configuration is still a work in progress
- The NLDAS surface data set is still a work in progress (will soon
  update this to use high-res PFTs)

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables):
- New xml variables:
  - CLM_CONFIGURATION (will eventually be renamed to CTSM_CONFIGURATION)
  - CLM_STRUCTURE (will eventually be renamed to CTSM_STRUCTURE)
- New namelist variable:
  - itmax_canopy_fluxes

Changes made to namelist defaults (e.g., changed parameter values):
No changes for existing CLM compsets; new defaults for NWP compsets

Changes to the datasets (e.g., parameter, surface or initial files):
No changes for existing grids; new surface dataset for nldas2 grid

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing:
- New tests covering NWP compsets, nldas grid and nldas datm forcing

Code reviewed by: Mike Barlage, Erik Kluzek


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - ok

    tests pass; namelists change as expected

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne ---- pass
    hobart ------ pass

CTSM tag used for the baseline comparisons: ctsm1.0.dev037


Answer changes
--------------

Changes answers relative to baseline: NO


Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.):
- MOSART: release-cesm2.0.00 -> nldas-grid.n01_mosart1_0_31
- CIME: ctsm/ctsm1.0/cime5.7.9/n04 -> ctsm/ctsm1.0/cime5.7.9/n05

Pull Requests that document the changes (include PR ids):
- https://github.com/ESCOMP/ctsm/pull/682
- https://github.com/ESCOMP/ctsm/pull/685

===============================================================
===============================================================
Tag name:  ctsm1.0.dev037
Originator(s):  sacks (Bill Sacks)
Date: Mon Apr 29 16:11:47 MDT 2019
One-line Summary: Change year alignment for present-day I compsets

Purpose of changes
------------------

Change year alignment for present-day I compsets (typically, these
compsets have aliases beginning with "I2000"), in order to work better
for cases that are meant to replicate particular years.

(1) Start the model in year 2000 rather than year 1

(2) Change datm's year alignment so that model year 2000 uses
observations from year 2000, model year 2001 uses observations from year
2001, etc.

(This change was requested by Sean Swenson and supported by Dave
Lawrence.)


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions):
- Year-2000 compsets now start in model year 2000 rather than 1

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: cime changes reviewed by Erik Kluzek and others


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    hobart ------ ok

    ok means tests pass, year-2000 cases change answers as expected

CTSM tag used for the baseline comparisons: ctsm1.0.dev036


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: Present-day compsets (I2000, I1Pt; also I2010 once those compsets are added)
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      larger than roundoff/same climate

      This changes the year alignment of datm. Climatological averages
      should be the same as before, but the alignment of model year to
      datm year changes.

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.):
- cime: ctsm/ctsm1.0/cime5.7.9/n03 -> ctsm/ctsm1.0/cime5.7.9/n04

Pull Requests that document the changes (include PR ids):
- cime changes documented in https://github.com/ESMCI/cime/pull/3093

===============================================================
===============================================================
Tag name:  ctsm1.0.dev036
Originator(s):  erik (Erik Kluzek,UCAR/TSS,303-497-1326)
Date: Fri Apr 26 01:20:57 MDT 2019
One-line Summary: Fix carbon isotope bug that caused wrong answers for isotopes under transient land-use change

Purpose of changes
------------------

We currently have a bug so that for transient land-use change cases answers are different starting in soil
carbon when Carbon isotopes (use_c13, or use_c14) are on versus off. Answers are identical if there is no
land-use change. The bug does cause bulk Carbon (C12) to be slightly different, but qualitatively the same.
There is a significant impact to the Carbon isotope simulation however. The bug was causing changes in isotopic
pools to be directed to the bulk Carbon. Because, isotopes are so much smaller than bulk Carbon, this doesn't 
cause a marked difference in the bulk Carbon simulation. But, the lack of the change in the isotopic pool
does have a meaningful impact on the simulation of the Carbon isotopic fields.

Also added a general test for this and specific one to catch this case. And did some updates on documentation
files, and a small bug fix for run_sys_tests.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): #692 #675 #686
 #675 -- Answers differ when carbon isotopes are on under transient cases
 #686 -- Add a system test to make sure turning on carbon isotopes doesn't change answers
 #692 -- run_sys_test issue

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[X] clm5_0

[X] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): None

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): None

Changes made to namelist defaults (e.g., changed parameter values): None

Changes to the datasets (e.g., parameter, surface or initial files): None

Substantial timing or memory changes: None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
Caveats for developers (e.g., code that is duplicated that requires double maintenance): None

Changes to tests or testing: Added LCISO test type and a 13 month Hist test to make sure carbon isotopes
                             don't cause a change to answers

Code reviewed by: dlawren, billsacks, klindsey, bishtgautum

CTSM testing: regular

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS

  tools-tests (test/tools):

    cheyenne - PASS

  PTCLM testing (tools/shared/PTCLM/test):

     cheyenne - PASS (compare tests fail, because of namelist updates)

  regular tests (aux_clm):

    cheyenne ---- OK
    hobart ------ OK

CTSM tag used for the baseline comparisons: ctsm1.0.dev035


Answer changes
--------------

Changes answers relative to baseline: Only for Carbon isotopes on under transient land-use change!

  Summarize any changes to answers, i.e.,
    - what code configurations: Bgc with use_c13 or use_c14 T and with a transient land-use change after each year boundary
    - what platforms/compilers: All
    - nature of change: same climate for bulk Carbon, isotopic Carbon is quite different for regions with land-use change

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: 

   URL for LMWG diagnostics output used to validate new climate:
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): cime
  cime from -- ctsm/ctsm1.0/cime5.7.9/n01 to ctsm/ctsm1.0/cime5.7.9/n03
    cime update allows the new test to work and starts adding NLDAS grid in

Pull Requests that document the changes (include PR ids): #696 #694
(https://github.com/ESCOMP/ctsm/pull)
  #696 -- run_sys_test small fix
  #694 -- Carbon isotope fix

===============================================================
===============================================================
Tag name:  ctsm1.0.dev035
Originator(s):  sacks (Bill Sacks)
Date: Mon Apr 22 20:17:40 MDT 2019
One-line Summary: Change h2ocan to a purely diagnostic variable

Purpose of changes
------------------

Previously, h2ocan_patch was maintained through the driver loop as the
sum of liqcan_patch and snocan_patch. However, there were places where
it was updated separately (though consistently) - i.e., it was not
simply a matter of 'h2ocan = liqcan + snocan'. This made the code
more complex, especially for the upcoming addition of water tracers.

This tag changes h2ocan_patch to be a purely diagnostic variable, set
once at the end of the driver loop. Prognostic equations that used to
use h2ocan now use liqcan and snocan, either directly or indirectly via
a temporary variable set equal to the current value of liqcan+snocan.

This tag also completely removes support for snowveg_flag = 'OFF' in
order to clean up the code.

In addition, this tag renames the namelist variable snowveg_flag to
snowveg_affects_radiation (now a logical).

Finally, this tag includes a minor change to always compute fcansno,
even if snowveg_affects_radiation is false, in order to simplify the
code.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#199


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables):
- snowveg_flag renamed to snowveg_affects_radiation, and changed to a
  logical flag

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by:
- Answer-changing portion of these changes reviewed by Keith Oleson and
  (at least to some extent) Sean Swenson


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    hobart ------ ok

    ok means tests pass, answers change as expected

CTSM tag used for the baseline comparisons: ctsm1.0.dev034


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: all
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      roundoff

   If bitwise differences were observed, how did you show they were no worse
   than roundoff?

     I split these changes into two pieces: (1) an answer-changing piece
     where I made minimal changes to replace uses of h2ocan_patch with
     liqcan+snocan (via temporary h2ocan variables holding this sum),
     and (2) non-answer-changing code cleanup. (1) consisted of the
     first two commits on this branch (461cafea and f195c105). With the
     changes in (1), I added checks near each change to ensure that the
     new expression differed by no more than roundoff from the old
     (using an absolute cutoff of 1e-13). I ran the full test suite with
     these checks in place, and generated new baselines. I then compared
     the changes in (2) with the baselines from (1); this cleanup step
     was bit-for-bit except for roundoff-level changes in the
     now-diagnostic-only field H2OCAN.

     As noted in ESCOMP/ctsm#689, the changes in (1) were larger than I
     expected, presumably because initially roundoff-level differences
     can lead to divergent code behavior. The most obvious example of
     this is in the check of h2ocan > 0 in subroutine FracWet. I have
     found that, due to roundoff errors, it is possible for h2ocan_patch
     > 0 by roundoff, while (liqcan_patch + snocan_patch) <= 0 by
     roundoff. (Presumably the reverse could also be true, though I
     haven't seen that – but I haven't spent much time looking for it.)

     Soon, we plan to do a more rigorous test to double check that this
     change is not climate-changing.

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
- First part of changes in https://github.com/ESCOMP/ctsm/pull/689

===============================================================
===============================================================
Tag name:  ctsm1.0.dev034
Originator(s):  sacks (Bill Sacks)
Date: Sat Apr 20 06:11:23 MDT 2019
One-line Summary: Change clm4_5: Use Justin Perket snow on vegetation

Purpose of changes
------------------

This tag changes the behavior of clm4_5, making it now use Justin
Perket's snow on vegetation parameterization. This parameterization was
originally introduced in clm4_5_1_r112. At the time it was only used for
clm5_0 cases. This tag uses that parameterization for clm4_5 cases, too,
so that (like clm5_0) they use snowveg_flag = 'ON_RAD' (i.e., vegetation
snow canopy on with albedo influence).

This changes climate for clm4_5, mainly through albedo changes,
particularly in the shoulder seasons.

The reason for this change is to allow an upcoming cleanup of the
relationship between h2ocan, snocan and liqcan. (See
https://github.com/escomp/ctsm/issues/199 for details.)

This tag also no longer allows snowveg_flag = 'OFF'. However, code
cleanup is deferred to a follow-on tag, in order to separate the answer
changes in this tag from the no-greater-than-roundoff-level changes in
the follow-on tag.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- First step towards addressing ESCOMP/ctsm#199 (Clean up relationship
  between h2ocan, snocan and liqcan)


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[X] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions):
- Changes the scientific configuration of clm4_5 cases, as noted above

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values):
- For clm4_5: snowveg_flag = 'ON_RAD' rather than 'OFF'

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance):
- Need to clean up code involving snowveg_flag, liqcan, snocan and
  h2ocan. I will do this in a follow-on tag.

Changes to tests or testing:
- Added a test with snowveg_flag = 'ON'. (At least some of the code
  paths triggered by this setting were previously covered by clm45
  cases. Now that both clm45 and clm50 use snowveg_flag = 'ON_RAD',
  there was some uncovered code. This new test covers this code.)
- Changed long single-point tests to use Qian forcing: With the default
  GSWP3 forcing, the vast majority of the runtime was being spent in
  datm. In addition, I was getting repeated failures of
  ERS_Lm54_Mmpi-serial.1x1_numaIA.I2000Clm50BgcCropGs.cheyenne_intel.clm-cropMonthOutput,
  I think due to memory issues; I'm hopeful that changing the datm
  forcing will help with that.

Code reviewed by: self


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    hobart ------ ok

    ok means tests pass; NLCOMP and BASELINE failures for Clm45 cases,
    as expected

CTSM tag used for the baseline comparisons: ctsm1.0.dev033


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: Clm45 compsets
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 

      New climate (as in clm4_5_1_r112 for clm5_0); the main impact is
      through albedo changes, particularly in the shoulder seasons.

      Changes in clm4_5 cases are due to:

      (1) Using snowveg_flag = 'ON_RAD' rather than 'OFF'

      (2) Changed initial conditions for canopy water: Rather than using
          H2OCAN, which is the meaningful state variable on previous
          clm4_5 initial conditions files, now we use LIQCAN and
          SNOCAN. These may be 0, although at least some clm4_5 initial
          conditions files (including the two non-Fates out-of-the-box
          clm4_5 initial conditions files) have non-zero (though still
          reasonable) LIQCAN for reasons I can't understand.

      (Also changes answers for
      ERS_D_Ln9_P480x3.f19_g16.I2000Clm50SpGs.cheyenne_intel.clm-waccmx_offline,
      which uses a clm45 initial file.)

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev033
Originator(s):  slevis (Samuel Levis, SLevis Consulting LLC, 303-665-1310)
Date: Thu Apr 11 10:51:29 MDT 2019
One-line Summary: Limit landunit presence with thresholds set in the namelist

Purpose of changes
------------------

 Faster simulations. Motivated by NWP applications:
 - Introduce user-defined namelist parameters to use as thresholds
 above which landunit areas are kept and
 below which landunit areas are removed.
 - Introduce user-defined logical namelist parameter to collapse the up to
 three urban landunits to the dominant urban landunit.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): #581
    Fixes #581 -- Allow zeroing out special landunits if their area is below some threshold 


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations? No

Notes of particular relevance for users
---------------------------------------

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables):
 - New namelist variables with valid real values from 0._r8 to 100._r8, where
 0 means "do nothing" and other values determine the threshold above
 which to keep the landunit in a gridcell. The new variables are:
 toosmall_soil
 toosmall_crop
 toosmall_glacier
 toosmall_lake
 toosmall_wetland
 toosmall_urban
 - New logical namelist variable collapse_urban. If .true. then the up to
 three urban landunits collapse to the dominant urban landunit.

---------------------------------------------
Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------

Caveats for developers (e.g., code that is duplicated that requires double maintenance):
    There's a typo in the name for n_dom_landunits (missing the s) in bld/CLMBuildNamelist.pm
    build-namelist doesn't check for toosmall_* below 0 or above 100, but the code does

Changes to tests or testing: Yes
 New unit tests subroutine test_collapse_individual_lunits and
 test_collapse_all_lunit_opts_active in test_surfrdUtils.pf
 Modified standard suite test
 ERS_D_Ld10.f10_f10_musgs.IHistClm50BgcCropGs.cheyenne_intel.clm-collapse_to_dominant_decStart
 to
 ERS_D_Ld10.f10_f10_musgs.I2000Clm50BgcCropGs.cheyenne_intel.clm-rm_indiv_lunits_and_collapse_to_dom
 and introduced fatal error message when namelist parameters toosmall_* > 0 or
 n_dom_* > 0 in transient simulations

Code reviewed by: Erik Kluzek, Bill Sacks

CTSM testing: regular

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS (384 differences because of new namelist items)

  regular tests (aux_clm):

    cheyenne ---- ok
    hobart ------ ok

CTSM tag used for the baseline comparisons:
 cheyenne ctsm1.0.dev031
 hobart   ctsm1.0.dev032


Answer changes
--------------

Changes answers relative to baseline: No
 Note: Comparison to baseline gives NLCOMP failures because the new namelist
 parameters always appear in the namelist now, even when they have not been
 changed from their default value by the user.

Detailed list of changes
------------------------
 - Introduced call to new subroutine collapse_individual_lunits that uses
 the user-defined namelist variables listed above to determine whether
 a landunit gets kept or removed from a gridcell.
 - Introduced new call to collapse_to_dominant for when collapse_urban = .true.
 - Introduced error checks that do not allow coexistence of transient
 simulations and the toosmall_* variables OR the n_dom_pfts/n_dom_landunits
 variables OR the collapse_urban parameter.

Pull Requests that document the changes (include PR ids): #641
 https://github.com/ESCOMP/ctsm/pull/641
   #641 -- Ability to remove individual landunits for faster runs

===============================================================
===============================================================
Tag name:  ctsm1.0.dev032
Originator(s):  negins (Negin Sobhani,UCAR/CSEG,303-497-1224)
Date: Mon Apr  8 08:26:57 MDT 2019
One-line Summary: Fixing the balance check to check for possible errors over all columns/patches

Purpose of changes
------------------
Up until now, the BalanceCheckMod code in biogeophysics did not check all
the possible errors over all columns or patches with the error threshold for
aborting clm and it merely compared the last column/patch found (where
warning threshold is met) with the error threshold.
Therefore, there was always a possibility that the abort is not triggered
when it should have been. This issue was first mentioned in ctsm issue #55
for soil energy balance check, but a similar issue is also present for
other balance checks in BalanceCheckMod.F90 .

Below is the list of the balance checks with a similar issue:
1. Water balance check
2. Snow balance check
3. Solar radiation energy balance check
4. Longwave radiation energy balance check
5. Surface energy balance check
6. Soil energy balance check (Issue #55)

For remediating this issue, we used MAXVAL and MAXLOC Fortran intrinsic functions
to compare the largest error with the warning and error thresholds instead
of checking the last column or looping over all columns.


In addition, this PR also makes slight changes to ./run_sys_tests outputi/case
directory structure. Previously for each case in a test suite, there were 
two set of directories created by run_sys_tests as follows:
 * one directory which included run/ and bld/ and named as $SCRATCH/[case-name]....
 * The case directory which included logs , timings, CaseDocs/, and misc. scripts
     for ./case.build, ./case.submit... under $SCRATCH/tests-[testID]/[case-name]....

This made debugging confusing and populated many folders in user's scratch.
Now, for every case in a test, all directories related to that case
including bld/ and run/ directories are nested under the case directory under $SCRATCH/tests-[testID]/[case-name]....
This is now similar to ./create_test output folder structure.



Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm issue#55

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): None

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): None

Changes made to namelist defaults (e.g., changed parameter values): None

Changes to the datasets (e.g., parameter, surface or initial files): None

Substantial timing or memory changes: None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance):

Changes to tests or testing: 
 - ./run_sys_test is changed so that it nests bld/ and run/ directories under the case directory as $SCRATCH/tests-[testID]/[case-name]...
   The output folder structure now is similar to ./create_test for each case in a test.

Code reviewed by: Bill Sacks


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run 

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    hobart ------ ok


CTSM tag used for the baseline comparisons: ctsm1.0.dev031


Answer changes
--------------

Changes answers relative to baseline: No


Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): None

Pull Requests that document the changes (include PR ids):
https://github.com/ESCOMP/ctsm/pull/670

===============================================================
===============================================================
Tag name:  ctsm1.0.dev031
Originator(s):  sacks (Bill Sacks)
Date: Wed Mar 13 15:03:42 MDT 2019
One-line Summary: Subtract virtual states to reduce dynbal fluxes for transient glaciers

Purpose of changes
------------------

Up until now, when computing the dynbal correction (conservation) fluxes
for transient glacier columns, we have been (1) counting the mass and
energy in the ice column as if that is a real state, but on the other
hand (2) NOT accounting for the fact that glacier columns don't
represent the soil under glacier. These two issues work in opposite
directions, but (1) dominates, because there is much more ice (and
energy, I think) in the roughly 50 m of glacial ice than there is in a
typical 50 m soil column.

In discussing this issue with Bill Lipscomb, we came up with the idea of
subtracting some baseline value from each glacier column. I think that,
as long as we subtract the same baseline value throughout an entire
simulation for a given column, we will still conserve mass and energy
through dynamic landunit transitions.

So, here we subtract baseline values from glacier columns, accounting
for the two issues mentioned above: (1) we subtract the water and energy
in the glacier ice, because this is a virtual state in CTSM, and (2) we
add the water and energy from the vegetated column(s), to account for
the fact that we don't have an explicit representation of
soil-under-glacier (this carries the assumption that the
soil-under-glacier has the same state as the initial vegetated state in
that grid cell). We set these baselines in initialization, so they begin
equal to the cold start state. Water and ice in the glacial ice stay
fixed over the course of a simulation, so the cold start values should
be the same as the current values at any point in time. The heat content
of the glacial ice does change over time, but by subtracting this
baseline value, we can potentially reduce the dynbal sensible heat
fluxes (however, note that it's also possible that these sensible heat
fluxes could increase when subtracting the cold start value, if ice
temperatures are closer to 0 deg C than to the cold start value,
currently 250 K).

In addition, this introduces a new namelist flag,
reset_dynbal_baselines, which allows the user to reset these baselines
at some desired point in the simulation. I think that, in general, this
resetting would break conservation. But as long as it is done before the
onset of transient glaciers, I think this should be okay. In this way,
the user can minimize dynbal fluxes even further, by resetting the
baselines after the system has spun up. If the states haven't changed
much from this point to the point when glacier dynamics occur, then the
dynbal fluxes should now be very small. (See the documentation of this
flag in namelist_definition_ctsm.xml for more details.)

Without this change, dynbal fluxes for a grid cell that undergoes 100%
glaciation / deglaciation in a single year are around 50 m ice, 1 m
liquid water, and a few tens of W m-2 (where those quantities of ice and
water are dribbled throughout the year, so we end up with an ice flux of
about 1.5e-6 mm/s throughout the year; and the energy flux is applied
evenly throughout the year - i.e., tens of W m-2 every time step for a
year). With this change, and with reset_dynbal_baselines set at an
appropriate time, these fluxes can be reduced to close to zero.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Partially addresses ESCOMP/ctsm#274 (Dynamic landunits: improve water
  and energy conservation)

CIME Issues fixed (include issue #):
- Fixes three issues related to inputdata checksum:
  - ESCMI/cime#3033
  - ESCMI/cime#3034
  - ESCMI/cime#3036

Known bugs introduced in this tag (include github issue ID):
- ESCOMP/ctsm#659 (Subtract dynbal baselines from begwb and endwb)
  - This can't be fixed until we're okay having this answer change on
    master

Known bugs found since the previous tag (include github issue ID):
- ESCOMP/ctsm#658 (Methane should not depend on gridcell-level TWS)

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[X] clm5_0

[X] clm4_5

Significant answer changes for runs with transient glaciers.

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions):
- For runs with transient glaciers, careful thought should be given to
  when to set reset_dynbal_baselines
- Inconsistency between begwb/endwb and liq1/liq2/ice1/ice2 (see ESCOMP/ctsm#659)

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables):
- New namelist variable: reset_dynbal_baselines

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance):
- Inconsistency between begwb/endwb and liq1/liq2/ice1/ice2 (see ESCOMP/ctsm#659)

Changes to tests or testing:
- Tweaked a test with transient glacier areas, including adding this new
  flag to the test

Code reviewed by: Bill Lipscomb reviewed the conceptual ideas, but
nobody has reviewed the actual code


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    hobart ------ ok

    ok means tests pass, answers change as expected

CTSM tag used for the baseline comparisons: ctsm1.0.dev030


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: ALL, but different configurations show
      different levels of changes, as noted below
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      - new climate (at least regionally) for some configurations;
        roundoff for others: see below

    There are three levels of changes here:

    (1) For runs with transient glaciers, there are large answer changes
        in regions with changing glacier area. These answer changes are
        in the various dynbal fluxes - for liquid water and ice runoff,
        and for the dynbal sensible heat flux.

    (2) For runs with transient vegetation (Hist, Dv and Fates), there
        are roundoff-level changes in the dynbal fluxes. For liquid
        water and ice, these changes appear to only occur in grid cells
        where there is some glacier area (because now both the before
        and after gridcell water contents have changed by the same fixed
        amount, which changes the difference at the roundoff level). For
        heat, there are roundoff-level changes everywhere (because of
        changes in the order of operations).

    (3) All runs have large changes in the diagnostic variables,
        ICE_CONTENT1, LIQUID_CONTENT1 and HEAT_CONTENT1, over grid cells
        containing some glacier area.

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.):
- cime: cime5.7.9 -> ctsm/ctsm1.0/cime5.7.9/n01
  - Fixes issues related to inputdata checksum

Pull Requests that document the changes (include PR ids):
- https://github.com/ESCOMP/ctsm/pull/650

===============================================================
===============================================================
Tag name:  ctsm1.0.dev030
Originator(s):  sacks (Bill Sacks)
Date: Fri Mar  8 15:08:34 MST 2019
One-line Summary: Update CIME; hookup expected test fails

Purpose of changes
------------------

Two main changes:

(1) Update to latest cime (from ctsm/ctsm1.0/cime5.7.5/n01 to
    cime5.7.9). Some of the important changes are:
    - Fix for recent cheyenne system changes
    - New domain file for f19_g17 (changes answers at the roundoff
      level)
    - New domain file for T31_g37 (changes answers by greater than
      roundoff)

(2) In system tests: Annotate cs.status output with expected failures

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Fixes ESCOMP/ctsm#654

CIME Issues fixed (include issue #): (Many)


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none?
   The PFS test showed a significant speedup (170s in the previous tag,
   124s in this tag). However, some other tests showed either a slowdown
   or not much change. So I'm not sure there is a consistent pattern of
   speedup in this tag.

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: Erik Kluzek


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    hobart ------ ok

    ok means tests pass, some baselines failed as noted below

    Most testing done with cime at eb2eb704f - excluded PR #3030, which
    updated support for CAM-SE grids (but the changes in this PR were
    very limited). Ran SSP tests with the final version of cime (so I'd
    have a few tests covering the latest cime).

CTSM tag used for the baseline comparisons: ctsm1.0.dev029


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: f19_g17, T31_g37
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 

      - f19_g17: roundoff-level, presumably due to new domain file (some
        diffs grew to greater than roundoff-level: some methane
        variables in a long test -
        SMS_Lm13.f19_g17.I2000Clm50BgcCrop.cheyenne_intel.clm-cropMonthOutput;
        and many variables in test with flooding -
        ERP_P180x2_D.f19_g17.I2000Clm50SpRtmFl.cheyenne_intel.clm-default)

      - T31_g37: greater than roundoff-level, presumably due to new
        domain file from cime commit 4fcf592ee


   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.):
- cime: ctsm/ctsm1.0/cime5.7.5/n01 -> cime5.7.9

Pull Requests that document the changes (include PR ids):
- https://github.com/ESCOMP/ctsm/pull/602

===============================================================
===============================================================
Tag name:  ctsm1.0.dev029
Originator(s):  slevis (Samuel Levis, SLevis Consulting LLC, 303-665-1310)
Date: Tue Feb 26 23:42:39 MST 2019
One-line Summary: Collapse landunits to the N most dominant

Purpose of changes
------------------

 Allows the collapse to fewer landunits using the concept of 
 N most dominant for the purpose of faster runs, eg in the context
 of NWP (Numerical Weather Prediction).

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): #581


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations? No

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): None

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables):
 New namelist variable n_dom_landunits with valid integer values from 0 to
 max_lunit currently equal to 9, where 0 means “do nothing” and other
 values determine the number of active landunits when running the model.

Changes made to namelist defaults (e.g., changed parameter values): None

Changes to the datasets (e.g., parameter, surface or initial files): None

Substantial timing or memory changes: None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------

Caveats for developers (e.g., code that is duplicated that requires double maintenance):
   There is a question left in on whether the collapse call needs to also
   be moved to inside dynamic landunits reading.

Changes to tests or testing: Yes
 ERS_D_Ld10.f10_f10_musgs.IHistClm50BgcCropGs.cheyenne_intel.clm-collapse_to_dominant_decStart
 now replaces (by combining with) this test introduced in ctsm1.0.dev026
 ERS_D_Ld10.f10_f10_musgs.IHistClm50BgcCropGs.cheyenne_intel.clm-collapse_nat_pfts_decStart
 I added subroutine test_collapse_to_dom_landunits to test_surfrdUtils.pf
 to include new Unit Tests. In same file I renamed
 subroutine test_collapse_nat_pfts to test_collapse_to_dom_pfts
 to accurately describe the contents of the subroutine.

Code reviewed by: Erik Kluzek

CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS

  regular tests (aux_clm):

    cheyenne ---- OK
    hobart ------ OK

CTSM tag used for the baseline comparisons: 
 ctsm1.0.dev028


Answer changes
--------------

Changes answers relative to baseline: No

Detailed list of changes
------------------------

 Replaced existing subroutine collapse_nat_pfts with
 subroutine collapse_to_dominant which can collapse to the
 N dominant landunits or N dominant pfts by performing the
 same actions on the data.

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): None

Pull Requests that document the changes (include PR ids): #639
 https://github.com/ESCOMP/ctsm/pull/639

===============================================================
===============================================================
Tag name:  ctsm1.0.dev028
Originator(s):  sacks (Bill Sacks)
Date: Tue Feb 26 14:35:01 MST 2019
One-line Summary: Interpolate out-of-the-box initial conditions and remove expensive tests

Purpose of changes
------------------

Two main changes (plus some small additional changes):

1. Removed / reworked some expensive tests

2. Interpolated all out-of-the-box initial conditions, so that the
   out-of-the-box version is now compatible with our current
   configuration. The changes from before were (a) our standard
   configuration now uses the gx1v7 rather than gx1v6 land mask; (b)
   many inactive points are now absent in memory.

See https://github.com/ESCOMP/ctsm/pull/622 for details.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#312
- Partially addresses ESCOMP/ctsm#275 (just a bit)


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files):
- New finidat files

Substantial timing or memory changes:
- Faster initialization times for cases that no longer need to interpolate initial conditions

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in README.CHECKLIST.master_tags as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing:
- Removed or reworked expensive tests: see details in https://github.com/ESCOMP/ctsm/pull/622

Code reviewed by: Erik Kluzek


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    hobart ------ ok

    ok: tests pass, baselines fail as expected

CTSM tag used for the baseline comparisons: ctsm1.0.dev027


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: all configurations that still need use_init_interp = .true.
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      Not investigated carefully, but expected to be larger than roundoff/same climate

  Answers change for cases that still need to do interpolation from
  these out-of-the-box finidat files.

  I first confirmed that the new finidat files are good via:

  - Compared `ncdump -h` of old vs. new initial conditions files

  - Ran the following tests, with comparison against master; verified that
    these all pass and are bit-for-bit, that they use the new initial
    conditions files, and that they now have `use_init_interp` unset
    (whereas they used to have `use_init_interp` set to `.true.`); note
    that these cover 6 of the 7 new initial conditions files:

    ```
    SMS_Ld1.f09_g17.I1850Clm45BgcGs.cheyenne_intel.clm-default
    SMS_Ld1.f09_g17.I1850Clm45BgcCruGs.cheyenne_intel.clm-default
    SMS_Ld1.f09_g17.I1850Clm50Sp.cheyenne_intel.clm-default
    SMS_Ld1.f09_g17.I1850Clm50BgcCrop.cheyenne_intel.clm-ciso
    SMS_Ld1.f09_g17.I1850Clm50BgcCropCru.cheyenne_intel.clm-ciso
    SMS_Ld1.f09_g17.I1850Clm50SpCru.cheyenne_intel.clm-default
    ```

  - To cover the last new initial conditions file, ran the following test
    (with the same checks as above: passes and bit-for-bit, checked
    finidat, and checked `use_init_interp`), with `./xmlchange
    LND_TUNING_MODE=clm5_0_cam6.0` (for the baseline for this test, I set
    `init_interp_method = 'use_finidat_areas'`, which I also used in
    interpolating this initial conditions file):
    
    ```
    SMS_Ld1.f09_g17.I1850Clm50BgcCrop.cheyenne_intel
    ```

  Details on answer changes:

    As noted above, these changes are bit-for-bit for configurations
    that can now use the out-of-the-box initial conditions without
    interpolation. However, answers change for configurations that still
    require interpolation of the out-of-the-box initial conditions
    (e.g., because they are at a different resolution than the finidat
    file).

    I expected these answer changes around the Caspian Sea, due to the
    fact that we are now effectively interpolating twice from the
    original file: once to produce the new out-of-the-box file (which
    includes an interpolation from `gx1v6` to `gx1v7`, and so
    interpolates onto the Caspian Sea region) and once to go from
    `f09_g17` to the target resolution (rather than interpolating
    directly from `f09_g16` to the target resolution).

    However, when I ran a test with comparison to baselines -
    `ERP_Ld5.f19_g17.I1850Clm50Bgc.cheyenne_intel.clm-default` - I saw
    changes in regions further afield than simply around the Caspian
    Sea. See images in https://github.com/ESCOMP/ctsm/pull/622 for
    details. It's possible that the other differences are also due to
    the Caspian Sea change, but it's also possible that there is some
    other cause here that I don't understand. But Erik Kluzek and I
    agreed that it isn't worth trying to understand these other
    (isolated) differences.

    Also note: While most f09_g17 tests are bit-for-bit with the earlier
    tag (because now use_init_interp is .false. for those tests),
    `SMS_Ld2_D.f09_g17.I1850Clm50BgcCropCmip6.cheyenne_intel.clm-basic_interp`
    differs in some glc forcing fields for a few grid cells in
    Antarctica. This is perhaps not surprising, since this test needs to
    use interpolation to get fields for the Antarctica virtual columns;
    so, while I don't completely understand the cause of these answer
    changes, I suspect the root cause is similar to the isolated answer
    changes I saw sprinkled around the world for
    `ERP_Ld5.f19_g17.I1850Clm50Bgc.cheyenne_intel.clm-default`.

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
- https://github.com/ESCOMP/ctsm/pull/622

===============================================================
===============================================================
Tag name:  ctsm1.0.dev027
Originator(s):  negins (Negin Sobhani,UCAR/CSEG,303-497-1224)
Date: Tue Feb 19 12:57:12 MST 2019
One-line Summary: Non-constant time initialization for soil hydrology types are moved to SoilHydrologyType.F90. 

Purpose of changes
------------------

Initalization of time-varying `zwt`, `zwt_perched`, and `frost_table` are moved to `InitCold` in `SoilHydrologyType.F90`.

Previously, `zwt`, `zwt_perched`, and `frost_table` were initialized in `SoilHydrologyInitTimeConstMod.F90`, which caused confusions.

Bugs fixed or introduced
------------------------
Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#575 ("Some variables initialized in SoilHydrologyInitTimeConstMod are not time-constant")

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: Updated a test.  The test added in previous tag included decStart with transient glaciers, which would not work. Hence, this test is modified so the compset does not include G.
Test IHistClm50SpG is updated to IHistClm50Sp.

Code reviewed by:
Bill Sacks
Erik Kluzek

CTSM testing:


  build-namelist tests:

    cheyenne - not run 

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    hobart ------ ok

    ok means tests pass, except for the expected failures.

CTSM tag used for the baseline comparisons: 
ctsm1.0.dev026

Answer changes
-------------

Changes answers relative to baseline: no


Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.):

Pull Requests that document the changes (include PR ids):
PR #631 Soil hydrology initialization of time-constant variables moved to SoilHydrologyType
https://github.com/ESCOMP/ctsm/pull/631


===============================================================
===============================================================
Tag name:  ctsm1.0.dev026
Originator(s):  slevis (Samuel Levis, SLevis Consulting LLC, 303-665-1310)
Date:  Tue Feb  5 17:08:48 MST 2019
One-line Summary: Collapse unmanaged PFTs to the N most dominant

Purpose of changes
------------------

To run simulations fast for applications such as NWP (Numerical Weather
Prediction), allow collapsing natural PFT's to the number the user wants.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): 457


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions):

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables):
New namelist parameter n_dom_pfts with valid integer values from 0 to
14, where 0 means “do nothing” and other values determine the number
of active pfts when running the model

Changes made to namelist defaults (e.g., changed parameter values): None

Changes to the datasets (e.g., parameter, surface or initial files): None

Substantial timing or memory changes: None (unless new option invoked)

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in ../CTSMMasterChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: updated tests, added new test
Updated standard testing test  <test name="ERS_D_Ld10"
grid="f10_f10_musgs" compset="IHistClm50SpG"
testmods="clm/collapse_pfts_78_to_16_decStart_f10">  to include decStart
(December start). Confirmed that answers were bit-for-bit same with
baseline before introducing the change.
Added standard testing test  <test name="ERS_D_Ld10"
grid="f10_f10_musgs" compset="IHistClm50BgcCropGs"
testmods="clm/collapse_nat_pfts_decStart">  to test n_dom_pfts = 2 in a
transient vegetation case with December start.
Added subroutine test_collapse_nat_pfts to test_surfrdUtils.pf to
include new Unit Tests.

Code reviewed by:
Erik Kluzek
Bill Sacks

CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS

  regular tests (aux_clm):

    cheyenne ---- OK
    hobart ------ OK

CTSM tag used for the baseline comparisons: 
ctsm1.0.dev025

Answer changes
--------------

Changes answers relative to baseline: No (bit-for-bit)

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): None

Pull Requests that document the changes (include PR ids):
https://github.com/ESCOMP/ctsm/pull/588 -- NWP collapse pfts to the N most dominant
https://github.com/ESCOMP/ctsm/pull/583 -- Add routine to find indices of max k values in data

Added new subroutines collapse_nat_pfts and collapse_crop_var and called
them from surfrdMod.F90. The second of these subroutines is not needed
when collapsing unmanaged PFTs; however, I created it early in the PR
and decided to keep it for use soon with crop-related variables like
irrigation water and fertilizer.

Bill Sacks made changes to include subroutine find_k_max_indices for the
benefit of this PR (#583)

===============================================================
===============================================================
Tag name:  ctsm1.0.dev025
Originator(s): Keith Oleson, Bill Sacks
Date: Wed Jan 23 10:48:01 MST 2019
One-line Summary: History fields for vertically-resolved sums of soil C and N, and minor fixes

Purpose of changes
------------------

Main change is from Keith Oleson: Add history fields for
vertically-resolved sums of SOIL1C, SOIL2C, and SOIL3C for C12, C13,
C14, and similarly for N. New fields are SOILC_vr, C13_SOILC_vr, and
C14_SOILC_vr, and SOILN_vr. For runs that use the output_bgc usermods,
including cmip6 runs, we no longer output 'SOIL1C_vr', 'SOIL1N_vr',
'SOIL2C_vr', 'SOIL2N_vr', 'SOIL3C_vr', 'SOIL3N_vr'; instead we output
'SOILC_vr', 'SOILN_vr', and similarly for C isotopes.

Also minor fixes:
- Output cpl hist files in SSP test (resolves ESCOMP/ctsm#61)
- Remove FATES-related commented-out code in OzoneMod (this has been
  moved to https://github.com/ESCOMP/ctsm/issues/618)
- Minor tweak to run_sys_tests

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#61

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values):
- For runs that use the output_bgc usermods, including cmip6 runs, we no
  longer output 'SOIL1C_vr', 'SOIL1N_vr', 'SOIL2C_vr', 'SOIL2N_vr',
  'SOIL3C_vr', 'SOIL3N_vr'; instead we output 'SOILC_vr', 'SOILN_vr',
  and similarly for C isotopes.

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in ../CTSMMasterChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: Bill Sacks; Erik Kluzek reviewed the changes to the SSP test


CTSM testing:

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne ---- pass
    hobart ------ pass

CTSM tag used for the baseline comparisons: ctsm1.0.dev024


Answer changes
--------------

Changes answers relative to baseline: NO


Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
https://github.com/ESCOMP/ctsm/pull/551

===============================================================
===============================================================
Tag name:  ctsm1.0.dev024
Originator(s):  slevis (Samuel Levis,SLevis Consulting, LLC, 303-665-1310)
Date:  Mon Jan 14 11:07:04 MST 2019
One-line Summary: Remove unnecessary restart variables

Purpose of changes
------------------

Remove unnecessary restart variables to conserve on computing resources
and disk space


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): 285


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions):

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables):

Changes made to namelist defaults (e.g., changed parameter values):

Changes to the datasets (e.g., parameter, surface or initial files):

Substantial timing or memory changes:

Notes of particular relevance for developers:
Code reviewed by Bill Sacks.
Standard testing (cheyenne and hobart) OK
Unit testing PASS
---------------------------------------------

Caveats for developers (e.g., code that is duplicated that requires double maintenance):

Changes to tests or testing:

Code reviewed by: Bill Sacks

CTSM testing:
Regular testing (cheyenne and hobart) OK
Unit testing PASS

CTSM tag used for the baseline comparisons: 
ctsm1.0.dev023


Answer changes
--------------

Changes answers relative to baseline: NO (bfb)


Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.):

Pull Requests that document the changes (include PR ids):
https://github.com/ESCOMP/ctsm/pull/598

Variables removed from restart:
plant_nalloc
pot_f_nit_vr*
f_nit_vr*
root_depth
qflx_floodg
snounload
qflx_snofrz_lyr
sminn_to_plant_fun_patch

===============================================================
===============================================================
Tag name:  ctsm1.0.dev023
Originator(s):  sacks (Bill Sacks)
Date: Thu Jan 10 13:00:42 MST 2019
One-line Summary: Remove CLM4.0

Purpose of changes
------------------

Remove CLM4.0. This consists of removing code and other files that are
specific to CLM4.0, and doing some resulting cleanup of code that was
shared between CLM4.0 and later versions (primarily in build-namelist).

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves #455 (Remove CLM4.0)


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions):
- Can no longer run CLM4.0 cases

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables):
- CLM4.0 compsets no longer supported

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in ../CTSMMasterChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: Removed CLM4.0 tests

Code reviewed by: Erik Kluzek


CTSM testing:

  build-namelist tests:

    cheyenne - pass

  tools-tests (test/tools):

    cheyenne - pass

  PTCLM testing (tools/shared/PTCLM/test):

     cheyenne - ok

       Total number of tests             = 52
       Number of tests that PASS         = 51
       Number of tests that Fail         = 1
       Number of compare tests that PASS = 37
       Number of compare tests that Fail = 4
       Number of tests without compare   = 11

       The one failing test is:
       FAIL fail-no_towerdata.-d+/glade/p/cesmdata/cseg/inputdata++-s+RF-Bra+--debug+--pftgrid+--soilgrid.

       These same failures occurred in ctsm1.0.dev022 (including the
       compare tests that Fail). Erik Kluzek says these failures are
       okay.

  regular tests (aux_clm):

    cheyenne ---- pass
    hobart ------ pass

  Notes on testing:

  - Most testing done on b5380c7e8. Since then, updated to a version of
    cime that fixes cime's configure tool (but this isn't invoked in
    system tests, so I didn't rerun the system tests) and pointed cime
    to a tag rather than a hash.

  - For tools tests: In baseline, cherry-picked commit 148bc2158 into
    cime

CTSM tag used for the baseline comparisons: ctsm1.0.dev022


Answer changes
--------------

Changes answers relative to baseline: NO

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.):
- cime: points to branch tag off of cime5.7.5
  (ctsm/ctsm1.0/cime5.7.5/n01), with changes for:
  (1) Removal of support for CLM4.0 (equivalent to the changes in
      https://github.com/ESMCI/cime/pull/2968)

  (2) Fix for cime configure (cherry-picked 148bc2158)

Pull Requests that document the changes (include PR ids):
- https://github.com/ESCOMP/ctsm/pull/609 (Remove CLM4.0)
- https://github.com/ESCOMP/ctsm/pull/601 (Print git status and related
  info when running run_sys_tests)


===============================================================
===============================================================
Tag name:  ctsm1.0.dev022
Originator(s):  sacks (Bill Sacks)
Date: Tue Jan  8 11:01:38 MST 2019
One-line Summary: Set tracer version of irrigation fluxes

Purpose of changes
------------------

Set tracer version of irrigation fluxes.

This required a substantial rewrite of ApplyIrrigation (now renamed to
CalcIrrigationFluxes). The problem was that the code was written to
compute the total irrigation withdrawal from groundwater, then use this
to compute the application fluxes (drip/sprinkler), then later divide
this total withdrawal by layer. But for tracer fluxes to be computed
correctly, we needed to reorder this, so that the per-layer withdrawals
are determined before determining the application fluxes. This is
because the application flux needs to know the tracer concentrations of
the source, which requires knowing which layers the source is drawing
from. This was made more challenging because of the mix of patch-level
and column-level variables at play: irrigation demand is patch-level,
groundwater availability and extraction is column-level, but application
is back to patch-level.

In a somewhat-related change, I also reworked the passing of information
between soil hydrology (groundwater availability) and irrigation:
Previously, there was some near-duplicate code in
CalcAvailableUnconfinedAquifer (which was called prior to
ApplyIrrigation) and WithdrawGroundwaterIrrigation (which was called
after ApplyIrrigation). I have reworked the code to remove this
duplication, calling the new CalcIrrigWithdrawals in the midst of
CalcIrrigationFluxes. In doing so, I reconciled an accidental
discrepancy between the two original routines (see
https://github.com/escomp/ctsm/issues/595).


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#521 (Set tracer versions of fluxes set by
  ApplyIrrigation)
- Resolves ESCOMP/ctsm#593 (Generalize groundwater irrigation availability to
  handle multiple patches per column)
- Resolves ESCOMP/ctsm#595 (Inconsistency between CalcAvailableUnconfinedAquifer
  and WithdrawGroundwaterIrrigation)


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

[ ] clm4_0

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by:

- Sean Swenson reviewed the initial parts of the rework of
  ApplyIrrigation and the passing of information between soil hydrology
  and irrigation

CTSM testing:

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    hobart ------ ok

    ok means tests pass, expected baseline failures as noted below

  Additional manual testing:

  - Out of the box, the one-timestep tracer consistency test
    (SMS_D_Ln1.f10_f10_musgs.I2000Clm50BgcCropGs.hobart_nag.clm-tracer_consistency)
    passed when the new check was inserted after the irrigation call,
    even without the changes in this tag. Presumably this is because no
    irrigation was being done in the first time step of this test. To
    confirm that the changes here were both necessary and effective, I
    hacked the code to force irrigation of all types (surface and
    groundwater) on the first time step (see
    https://github.com/billsacks/ctsm/commit/d28a03145). I confirmed
    that this test, with those code hacks, failed before the changes in
    this tag, and passed with these changes in place.

  - I used a multi-step process to confirm that these changes are only
    roundoff-level different for groundwater irrigation (because
    feedbacks in the system result in
    SMS_D_Ld5.f10_f10_musgs.I2000Clm50BgcCrop.hobart_nag.clm-irrig_alternate
    looking greater-than-roundoff-level different from the baseline):

    (1) Confirmed that e507fe603 is only roundoff-level different from
        baseline using cprnc output from this test.

    (2) Confirmed that a4a1a626c is only roundoff-level different from
        e507fe603. This was the tricky part. I did this by introducing
        some temporary code that (a) computed some fluxes in both the
        old and new ways, (b) compared the old and new methods for each
        point and time step, confirming that they were no more than
        roundoff-level different, then (c) set the fluxes to the old
        method. I confirmed that this was bit-for-bit with e507fe603,
        and the checks for greater-than-roundoff-level differences
        between the old and new methods were never triggered. (See
        https://github.com/billsacks/ctsm/commit/7d23e9e and the earlier
        commits on that branch.)

    (3) Confirmed that remaining changes an the branch are bit-for-bit
        with a4a1a626c.

CTSM tag used for the baseline comparisons: ctsm1.0.dev021


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations:

      - Small answer changes when groundwater irrigation is enabled. In
        principle, could change answers by greater than roundoff due to
        fix of #595, but I didn't see any answer changes in my 7-month
        test due to that change. Other than that, just roundoff-level
        changes.

      - Answer changes when tracers are enabled.

      - Roundoff-level changes in the diagnostic field,
        QIRRIG_FROM_SURFACE, for many tests

    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      See notes under "what code configurations" for details.

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
https://github.com/ESCOMP/ctsm/pull/600

===============================================================
===============================================================
Tag name:  ctsm1.0.dev021
Originator(s):  mvr (Mathew Rothstein,UCAR/CSEG,303-497-1304)
Date: Wed Dec 26 16:29:06 MST 2018
One-line Summary: Added tracer ratio capability and included it in consistency checks

Purpose of changes
------------------

These changes are needed to support the implementation of water isotopes by ensuring
that isotope-related variables are being updated consistently throughout the model.
The consistency checks include the testing of a bulk tracer that should always be
equal to the bulk, and also test tracers that maintain a fixed ratio (other than 
one) to the bulk.  This commit includes fixes to the cold start initialization for 
the water tracers with ratios other than one.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): [If none, remove this line]
#541 Only do water tracer consistency checks on tracers that are
	supposed to maintain a fixed ratio
#459 Fix cold start initialization of water tracers
#508 Remove unused variables in new Water types
#357 Put in place Water isotope consistency checks
	


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

[ ] clm4_0

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions):  none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables):  none

Changes made to namelist defaults (e.g., changed parameter values):  none

Changes to the datasets (e.g., parameter, surface or initial files):  none

Substantial timing or memory changes:  none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in ../CTSMMasterChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance):  none

Changes to tests or testing:  added unit tests

Code reviewed by:  bill sacks


CTSM testing:


  build-namelist tests:

    cheyenne - not run 

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne ---- PASS
    hobart ------ PASS

CTSM tag used for the baseline comparisons:  ctsm1.0.dev020


Answer changes
--------------

Changes answers relative to baseline: no, b4b



Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): #599 Adding isotope 
	tracer ratios
(https://github.com/ESCOMP/ctsm/pull)

===============================================================
===============================================================
Tag name:  ctsm1.0.dev020
Originator(s):  Sean Swenson, Bill Sacks
Date: Mon Dec  3 11:51:24 MST 2018
One-line Summary: New options for irrigation and crop fsat

Purpose of changes
------------------

Introduce three new options:

(1) Ability to withdraw irrigation water from groundwater if not enough
    water is available from rivers. This is controlled via new namelist
    flag, use_groundwater_irrigation. Water can be withdrawn from both
    the unconfined (from the soil column) and confined (from wa)
    aquifers.

(2) Irrigation method: sprinkler (above canopy) vs. drip (below
    canopy). This can be set on a per-crop and per-gridcell basis on the
    surface dataset, but out-of-the-box support for creating the
    necessary surface dataset field is not yet in place (see
    ESCOMP/ctsm#565). For now, it can be controlled globally via a new
    namelist flag, irrig_method_default. The default is drip, which was
    what we were previously using implicitly.

(3) Set crop fsat to zero. This is controlled by a new namelist option,
    crop_fsat_equals_zero.

Default behavior is the same as before for all three options.

Also:

- If use_aquifer_layer is false (which is the default for CLM50), no
  longer reset wa_col every time step

- Adds indices to vector history files giving column, landunit and
  gridcell indices for each patch, etc. (Resolves ESCOMP/ctsm#81:
  "Restart files are different for CLM when run over different number of
  tasks" (issue name is a misnomer of the remaining to-dos in that
  issue.)

- Writes 3d time-constant fields on first history file for all tapes,
  not just the first

- Small performance improvements to irrigation

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Resolves ESCOMP/ctsm#81 ("Restart files are different for CLM when run
  over different number of tasks", but that name was a misnomer for the
  remaining work needed on the issue)

Known bugs introduced in this tag (include github issue ID):
- ESCOMP/ctsm#565: Surface dataset enhancements needed for irrigation method


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

[ ] clm4_0

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables):
- New namelist variables:
  - use_groundwater_irrigation
  - irrig_method_default
  - crop_fsat_equals_zero
- Partial support for new surface dataset field: irrigation_method

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in ../CTSMMasterChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance):

Changes to tests or testing:
- Added two tests of the new irrigation options
- Added an _includes directory to hold things that should be included by
  testmods (not used directly)


Code reviewed by: Bill Sacks and Sean Swenson reviewed each other's code


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - ok

    Tests pass, namelists differ as expected

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    hobart ------ pass

    ok: tests pass, some answers change as noted below

CTSM tag used for the baseline comparisons: ctsm1.0.dev019


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: Some CLM50 cases that are either cold start
      or (unusually) use a CLM45 initial conditions file; it's possible
      that non-cold-start configurations would see differences rarely.
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      Larger than roundoff, same climate

      Differences arise due to no longer resetting wa_col to
      aquifer_water_baseline every time step in BeginWaterBalanceSingle
      if use_aquifer_layer is false (which is the default for
      CLM50). (This change is needed to conserve water properly with the
      new groundwater-based irrigation.) For the most part, wa_col
      wasn't being changed when use_aquifer_layer is false (so no longer
      resetting it has no effect if groundwater irrigation isn't being
      used), but the one exception is the "work around of the negative
      liquid water" added by Jinyun Tang in subroutine SoilWater (dated
      Jan 14, 2015), which can be exercised when use_flexibleCN is
      true. Sean Swenson did some experimentation, and found that the
      code that updates wa_col is only exercised in the first time step
      of a cold start run, and that finding is borne out in the test
      suite results. However, it's possible that there are rare cases
      when this is exercised later in a run.

      I have confirmed that this branch is bit-for-bit with the baseline
      if I revert the addition of the use_aquifer_layer conditional
      around the wa_col resetting in
      BeginWaterBalanceSingle. Alternatively, nearly all tests are
      bit-for-bit with the baseline if I remove the one line updating
      wa_col in subroutine SoilWater (and also remove the endrun for
      water balance errors from BalanceCheck); the one exception is the
      waccmx_offline test, which uses a CLM45 initial conditions file in
      a CLM50 configuration.

      The following tests had baseline failures:

      Cold start tests:

         ERP_D_Ld5.f10_f10_musgs.IHistClm50BgcCrop.cheyenne_intel.clm-allActive
         ERP_D_P36x2_Ld3.f10_f10_musgs.I2000Clm50BgcCrop.cheyenne_intel.clm-cropColdStart
         ERP_P180x2_D_Ld5.f19_g17.I2000Clm50BgcDvCrop.cheyenne_intel.clm-crop
         ERP_P36x2_Lm25.f10_f10_musgs.I2000Clm50BgcDvCrop.cheyenne_intel.clm-monthly
         ERP_P72x2_Lm25.f10_f10_musgs.I2000Clm50BgcDvCrop.cheyenne_intel.clm-monthly

      Test that uses a CLM45 initial conditions file in a CLM50
      configuration:

         ERS_D_Ln9_P480x3.f19_g16.I2000Clm50SpGs.cheyenne_intel.clm-waccmx_offline

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
https://github.com/ESCOMP/ctsm/pull/523

===============================================================
===============================================================
Tag name:  ctsm1.0.dev019
Originator(s):  sacks (Bill Sacks)
Date: Fri Nov 30 13:36:57 MST 2018
One-line Summary: Rework cold start initialization of wa and zwt

Purpose of changes
------------------

Rework cold start initialization of wa and zwt when use_aquifer_layer is
false to reduce answer changes in upcoming groundwater_irrigation
branch.

In the groundwater_irrigation branch
(https://github.com/ESCOMP/ctsm/pull/523), Sean Swenson has stopped
resetting wa_col each time step if use_aquifer_layer is false. However,
this leads to having a substantially different value of wa_col when
use_aquifer_layer is false: previously, it was reset to
aquifer_water_baseline each time step, but with Sean's changes, it stays
close to its initial values, which have been 4000 in most places. This
tag changes the initial values to match the value it was being reset to,
so it simply starts at aquifer_water_baseline - so the
every-time-step-resetting to aquifer_water_baseline can be removed
without massively changing the value of wa_col.

In addition to changing the cold start initialization of wa_col, we are
also changing the cold start initialization of zwt in this case where
use_aquifer_layer is false: The old initialization of zwt wouldn't work
as intended now that we have changed the initial value of wa_col; Sean
Swenson suggested this new initialization method instead. This
initialization to zi(c,nbedrock(c)) is correct if there are no saturated
layers, and close enough for a decent cold start even if there are.


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

[ ] clm4_0

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in ../CTSMMasterChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: Sean Swenson


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    hobart ------ ok

    ok means tests pass, answers change as expected for some cases

CTSM tag used for the baseline comparisons: ctsm1.0.dev018


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: CLM50 cold start or transient (Hist) cases
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 

      Larger than roundoff, same climate (not investigated completely
      rigorously through long simulations, but Sean Swenson and Bill
      Sacks are both pretty confident that the resulting changes will be
      small, partly based on difference statistics from the test suite,
      and partly based on the fact that the only change in this tag is
      in cold start initialization of some values).

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
https://github.com/ESCOMP/ctsm/pull/577

===============================================================
===============================================================
Tag name:  ctsm1.0.dev018
Originator(s):  sacks (Bill Sacks)
Date: Thu Nov 29 16:03:50 MST 2018
One-line Summary: Water tracer updates for initial things in driver loop

Purpose of changes
------------------

Update water tracers for initial stuff done in driver loop. This includes
atm2lnd forcings (non-downscaled and downscaled), balance check initialization,
and dyn subgrid updates.

Broadly speaking, the changes here are:

(1) Reworked WaterType to make it easier / more robust for other code to loop
    over tracers or bulk+tracers

(2) The most interesting changes are probably the code to update the atm2lnd
    water tracers (in Wateratm2lndType.F90 and WaterTracerUtils.F90)

(3) In various other places, do some infrastructurey stuff (initializing water
    balance, doing dyn subgrid stuff) for tracers as well as bulk

(4) Supporting unit tests and unit test infrastructure


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
Resolves ESCOMP/ctsm#487
Resolves ESCOMP/ctsm#488
Resolves ESCOMP/ctsm#489


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

[ ] clm4_0

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in ../CTSMMasterChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance):
- We have chosen not to set all water tracers as soon as possible, but instead
  to do these tracer settings later in the driver loop. This keeps the driver
  loop cleaner, but means that you cannot arbitrarily sprinkle calls to
  TracerConsistencyCheck throughout the driver. Specifically for this tag: the
  non-downscaled, gridcell-level atm2lnd water tracers are not updated until
  after the call to downscale_forcings, so tracer consistency checks before that
  point would fail.

Changes to tests or testing:
- Added a PFS test

Code reviewed by: Portions of the design (and possibly code) have been reviewed
by Mat Rothstein, David Noone and Mariana Vertenstein


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    hobart ------ ok

    ok means tests pass, some answers change as expected

CTSM tag used for the baseline comparisons: ctsm1.0.dev017


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: many
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      roundoff-level changes in sensible heat flux from precip conversion due to
      refactoring this calculation; everything else bit-for-bit

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? via summarize_cprnc_diffs to see differences in the test suite

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
https://github.com/ESCOMP/ctsm/pull/572

===============================================================
===============================================================
Tag name:  ctsm1.0.dev017
Originator(s):  slevis (Samuel Levis, Slevis Consulting LLC,303-665-1310)
Date: Wed Nov 28 14:27:50 MST 2018
One-line Summary: Merge the collapse2gencrop branch

Purpose of changes
------------------

These changes allow the model to not need to read 16-pft
datasets and rather read 78-pft datasets. The 78-pft datasets were
developed for use with prognostic crops originally. The current changes
allow the model to use the 78-pft datasets in 16-pft runs by
collapsing the crop pfts (cfts) from specific types to the model's
generic crop types. The changes are generic so that the model may still
read 16-pft datasets for 16-pft runs. Ultimately these changes will
permit users to run with any number of pfts, while the ctsm group
maintains a single set of input pft data.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): Fixes #509 (partial)
 #509 -- irrigate in 1850 is off for runs with use_crop but on for those without

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

[ ] clm5_0

[ ] clm4_5

[ ] clm4_0

Notes of particular relevance for users
---------------------------------------

Changes made to namelist defaults (e.g., changed parameter values):
- maxpatch_pft made obsolete
- nnegcrit was increased because of failure in the TRENDY simulations.
This was done in the source, rather than the namelist but does have an
effect on user behavior.

Substantial timing or memory changes: No

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in ../CTSMMasterChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance):

   The "irrigate" attribute should now be removed from all fsurdat files for namelist generation.
   Thus irrigate on or off is now an option for non-crop cases.


Renamed parameters:
maxpatch_pft is obsolete; remove from namelist-related scripts in future PR
numpft + 1 ---> maxsoil_patches
maxpatch_pft ---> maxsoil_patches in CNDV
numpft ---> maxveg
numpft_ed ---> maxveg_fates

Several modules were made default private. The list of variables in them
that are public are explicitly listed now:
LakeCon.F90
clm_initializeMod.F90
clm_instMod.F90 
clm_varpar.F90
pftconMod.F90
subgridAveMod.F90

Changes to tests or testing:
New test   <test name="ERS_D_Ld10" grid="f10_f10_musgs" compset="IHistClm50SpG" testmods="clm/collapse_pfts_78_to_16_f10">
New unit tests in test_surfrdUtils.pf, all containing the prefix
test_collapse_crop_types_*

Code reviewed by:
Erik Kluzek and Bill Sacks

CTSM testing:
Regular

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  regular tests (aux_clm):

    cheyenne_intel ---- OK
    cheyenne_gnu ------ OK
    hobart_nag -------- PASS
    hobart_pgi -------- OK
    hobart_intel ------ PASS

CTSM tag used for the baseline comparisons: ctsm1.0.dev016

Answer changes
--------------

Changes answers relative to baseline: NO


Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): None

Pull Requests that document the changes (include PR ids): #483

Sam Levis cloned Erik Kluzek's collapse2gencrop branch and then
tested, corrected errors, resolved conflicts, added unit tests, while
updating to newer tags as needed.


===============================================================
===============================================================
Tag name:  ctsm1.0.dev016
Originator(s):  sacks (Bill Sacks)
Date: Thu Nov  1 07:27:48 MDT 2018
One-line Summary: Update cime, fix FATES DEBUG token, add script to easily run system tests

Purpose of changes
------------------

(1) Update to latest version of cime master

(2) Update FATES with a minor change: change DEBUG to debug, to allow
    working with a preprocessor-defined DEBUG token

(3) Add a script (run_sys_tests) that allows easily running all system
    tests (see
    https://github.com/ESCOMP/ctsm/wiki/System-Testing-Guide#running-test-suites-with-the-run-sys-tests-wrapper
    for details)

(4) As part of (3), start work on a CTSM python library and associated
    test infrastructure.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Fixes ESCOMP/ctsm#535 (Run Fortran unit tests as part of create_test)

CIME Issues fixed (include issue #):
- Various - see CIME ChangeLog for details


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[X] clm5_0

[X] clm4_5

[X] clm4_0

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in ../CTSMMasterChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing:
- New run_sys_tests wrapper
- Fortran unit tests now run as part of aux_clm

Code reviewed by: Erik Kluzek


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  tools-tests (test/tools):

    cheyenne - not run

  PTCLM testing (tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne ---- ok
    hobart ------ ok

    ok means tests pass, baselines fail as expected

CTSM tag used for the baseline comparisons: ctsm1.0.dev015


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: All I compsets (except spinup test that uses cplhist datm forcing)
    - what platforms/compilers: All
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      Potentially new climate

  Changes answers for I compsets due to datm update as part of the cime
  update: new presaero and CO2 datasets
  (https://github.com/esmci/cime/pull/2828)

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.):
- cime: cime5.7.3 -> cime5.7.5
- fates: fates_s1.8.1_a3.0.0 -> fates_s1.8.1_a3.0.0_rev2

Pull Requests that document the changes (include PR ids):
- https://github.com/escomp/ctsm/pull/493 (Wrapper to system tests, and
  start of a ctsm python library)

===============================================================
===============================================================
Tag name:  ctsm1.0.dev015
Originator(s):  sacks (Bill Sacks)
Date: Sun Oct 28 14:29:30 MDT 2018
One-line Summary: CMIP6 compset modifiers, usermods for typical output, and other output enhancements

Purpose of changes
------------------

(1) Support %BGC-CROP-CMIP6DECK and %BGC-CROP-CMIP6WACCMDECK compset
    modifiers, so that we can turn on the necessary options
    (output-related and others) via new CMIP6-specific compsets.

(2) Turn on carbon isotopes in CMIP6 runs (from Erik Kluzek)

(3) Remove setting of CCSM_BGC=CO2A in the cmip6 usermods

(4) Add usermods directories for getting typical extra output that's
    wanted in many cases: output_crop, output_crop_highfreq, output_bgc,
    output_bgc_highfreq, output_sp, and output_sp_highfreq. These can be
    enabled by adding something like '--user-mods-dir output_crop' on
    the create_newcase line (that short-hand works for an I compset; for
    F or B compsets, you need to provide the full path to the usermod
    directory).

(4) Allow holes in the number of history tapes. Holes are cases where,
    for example, we have h0, h1 and h3 tapes, but no h2 tape (because
    there are no fields on the h2 tape). (This is needed for (3).)

(5) Fix reading and writing of 1-d logical global arrays. This fixes
    ESCOMP/ctsm#24 for real (rather than just preventing an attempt to
    read/write 1-d logical arrays, as was done in the previous 'fix').

(6) Add C13_NBP and C14_NBP diagnostic fields (from Keith Oleson)

(7) Make a bunch of carbon isotope diagnostic fields inactive by default

(8) Don't allow interpolation (use_init_interp) from a case without
    carbon isotopes to a case with carbon isotopes: Due to
    https://github.com/ESCOMP/ctsm/issues/67, interpolation from a case
    without carbon isotopes to a case with carbon isotopes yields
    incorrect initialization values for the carbon isotopes. Now that
    we're turning carbon isotopes on via some semi-out-of-the-box
    usermods (for cmip6), it is becoming more important to check to make
    sure someone doesn't shoot themselves in the foot this way.

(9) Add tests of the new output usermods as well as of the CMIP6 compset
    modifiers

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Fixes #24 for real (ncd_io_1d_log_glob is broken)
- Fixes #529 (Organize usermods_dirs to facilitate running cases with
  the right output)


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

[ ] clm4_0

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables):
- New namelist variable, just for testing purposes:
  for_testing_allow_interp_non_ciso_to_ciso. This bypasses an error
  check, and should only be used in tests.

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in ../CTSMMasterChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing:
- Reworked tests of cmip6_deck usermods to use the new
  %BGC-CROP-CMIP6DECK compset modifier, and changed f09_g16 to f09_g17
- Added a test of the %BGC-CROP-CMIP6WACCMDECK compset modifier
- Added tests of output_crop_highfreq, output_bgc_highfreq and
  output_sp_highfreq usermods directories
- Note that there are NO tests covering the cmip6_evolving_icesheet
  usermods: this usermod directory adds very little beyond cmip6_deck,
  so it didn't seem worth its own test

Code reviewed by: Erik Kluzek


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  unit-tests (components/clm/src):

    cheyenne - pass

  tools-tests (components/clm/test/tools):

    cheyenne - not run

  PTCLM testing (components/clm/tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne_intel ---- pass
    cheyenne_gnu ------ pass
    hobart_nag -------- pass
    hobart_pgi -------- pass
    hobart_intel ------ pass

    ERI_D_Ld9.T31_g37.I2000Clm50Sp.cheyenne_intel.clm-SNICARFRC
    initially failed COMPARE_base_hybrid, COMPARE_base_rest and BASELINE
    comparisons. I reran it and it passed.

    Manually compared all history files from
    ERS_Ly3.f10_f10_musgs.I1850Clm50BgcCropCmip6.cheyenne_intel.clm-basic
    with baseline
    ERS_Ly3.f10_f10_musgs.I1850Clm50BgcCrop.cheyenne_intel.clm-cmip6. They
    were all identical. Also compared all history files from
    SMS_Ld5_D.f09_g17.I1850Clm50BgcCropCmip6.cheyenne_intel.clm-basic
    with baseline
    SMS_Ld5_D.f09_g17.I1850Clm50BgcCrop.cheyenne_intel.clm-cmip6. They
    were all identical (but note that this basically just compared the
    cpl.hi file: the test was too short to produce monthly files.)

CTSM tag used for the baseline comparisons: ctsm1.0.dev014


Answer changes
--------------

Changes answers relative to baseline: NO


Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
- https://github.com/ESCOMP/ctsm/pull/536 - Allow holes in the number of
  history tapes and reorganize cmip6 usermods (main PR containing all of
  these changes)
- https://github.com/ESCOMP/ctsm/pull/527 - Add carbon_isotope user-mod
  directory to turn on c13 and c14
- https://github.com/ESCOMP/ctsm/pull/539 - Support a %CMIP6DECK compset
  modifier

===============================================================
===============================================================
Tag name:  ctsm1.0.dev014
Originator(s):  sacks (Bill Sacks)
Date: Fri Oct 26 06:20:34 MDT 2018
One-line Summary: Miscellaneous minor, bit-for-bit bug fixes

Purpose of changes
------------------

Four miscellaneous minor, bit-for-bit bug fixes:

(1) Py3 pylint check and address cime issue ESMCI/cime#2822 (from Jim
    Edwards: ESCOMP/ctsm#526)

(2) Change uppercase DEBUG variables to lowercase debug (requested by
    Jim Edwards to avoid conflicting with the DEBUG CPP token)
    (Fixes ESCOMP/ctsm#534)

(3) Remove unnecessary line of code in LunaMod.F90 that was causing
    problems with some compilers due to an uninitialized variable
    (Fixes ESCOMP/ctsm#322)

(4) Add r8 to 0 constant to fix build issue with XLF compiler (from Jim
    Edwards: ESCOMP/ctsm#531)


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Fixes ESCOMP/ctsm#322 (ERS_D_Ld5.f19_g16.I2000Clm50BgcCruGs run FAIL (intel))
- Fixes ESCOMP/ctsm#534 (Rename DEBUG to debug in a few places)


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

[ ] clm4_0

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in ../CTSMMasterChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance):

Changes to tests or testing: none

Code reviewed by: different pieces reviewed by different people


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  unit-tests (components/clm/src):

    cheyenne - pass

  tools-tests (components/clm/test/tools):

    cheyenne - not run

  PTCLM testing (components/clm/tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne_intel ---- ok
    cheyenne_gnu ------ ok
    hobart_nag -------- ok
    hobart_pgi -------- ok
    hobart_intel ------ ok

    ok means tests and baseline comparisons pass. There were unexpected
    NLCOMP diffs. From spot-checking a few tests, I see the following:

    
    (1) On both cheyenne and hobart: Diffs in logfile. This looks like a
        problem in cime: it says:

        BASE: logfile = 'rof.log.136342.hobart.cgd.ucar.edu
        COMP: logfile = 'rof.log

    (2) On hobart: Diffs in pio_typename: netcdf vs. pnetcdf (says that
        new uses pnetcdf): but when I compare files by hand, they look
        the same in this respect (both baseline and new use netcdf), so
        maybe this is due to a problem in the timing of when nlcomp is
        run?

    Since these both look like problems in the comparison script rather
    than in the tag, I'm letting these go, but will open a cime issue if
    we see this again.


CTSM tag used for the baseline comparisons: ctsm1.0.dev013


Answer changes
--------------

Changes answers relative to baseline: NO


Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
- https://github.com/ESCOMP/ctsm/pull/531 (fix build issue with xlf compiler)
- https://github.com/ESCOMP/ctsm/pull/526 (fix cime issue 2822 and pylint chk)

===============================================================
===============================================================
Tag name:  ctsm1.0.dev013
Originator(s):  erik (Erik Kluzek)/slevisconsulting
Date: Thu Oct 25 18:09:47 MDT 2018
One-line Summary: Fix the fact that transient Bgc and SP cases had constant crop area in time

Purpose of changes
------------------

Fix transient non-crop cases that had constant crop area so that crop area will change in time
(and hence natural veg area will also change corresponding to it).

Also bring in changes to update mksurfdata_map to handle SSP-RCP future scenarios. Right now
it can handle SSP5-8.5 out of the box. Also add a new test for that.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):  #538
   #538 -- In transient pft simulations with use_crop=.false., %crop does not advance from 1850 values

Known bugs found since the previous tag (include github issue ID): #545
   #545 -- Antarctica ice shelves are being treated as wetlands rather than glaciers


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[X] clm5_0

[X] clm4_5

[ ] clm4_0

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): A few more namelist checks
   Added a check to make sure do_transient_pfts was the same as do_transient_crops
   Don't allow use_fertilizer for non-crop case
   namelist checks that create_crop_landunit=T except for FATES

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables):
   New -ssp_rcp option to mksurfdata.pl

Changes made to namelist defaults (e.g., changed parameter values): None

Changes to the datasets (e.g., parameter, surface or initial files): New mksrfpft datasets in XML for SSP5-8.5

Substantial timing or memory changes: None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in ../CTSMMasterChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance):
   Now, do_transient_pfts and do_transient_crops only work in unison, and hence
   should be simplified to one logical called do_transient_veg

Changes to tests or testing: Add new SSP5-8.5 mksurf test

Code reviewed by: self, slevisconsulting, billsacks

CTSM testing: regular, tools

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS (18 of the transient test namelists are different to baseline as expected)

  unit-tests (components/clm/src):

    cheyenne - PASS

  tools-tests (components/clm/test/tools):

    cheyenne - OK (PTCLM tests fail)

  regular tests (aux_clm):

    cheyenne_intel ---- OK
    cheyenne_gnu ------ OK
    hobart_nag -------- OK
    hobart_pgi -------- OK
    hobart_intel ------ OK

CTSM tag used for the baseline comparisons:  ctsm1.0.dev012


Answer changes
--------------

Changes answers relative to baseline: Yes!

  Summarize any changes to answers, i.e.,
    - what code configurations: clm4_5/clm5_0 transient simulations without Crop (either Bgc or SP)
    - what platforms/compilers: All
    - nature of change: crop areas increase in time and impact simulations

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: dll/clm50_r267_1deg_GSWP3V1_iso_hist_nocrop_transientfix

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): None

Pull Requests that document the changes (include PR ids): #546, #540
(https://github.com/ESCOMP/ctsm/pull)

   #546 -- Add in ability for mksurfdata_map to handle ssp_rcp scenarios and specifically SSP5-8.5
   $540 -- Transient PFT issue

===============================================================
===============================================================
Tag name:  ctsm1.0.dev012
Originator(s):  erik (Erik Kluzek)
Date: Sat Sep 29 11:49:35 MDT 2018
One-line Summary: Add snow-free fields for snowmip, fix several issues

Purpose of changes
------------------

Bring in new diagnostic fields added by Justin Perket, Sean Swenson and Mark Flanner
for Snow-MIP. Most of those are fields that represent "Snow Free" data.

Also bring in fixes for a list of issues. Add handling of the new CO2 which includes
both latitude-band and global average versions. Add some changes to make it easier
for input data processing including NOT doing the slow 1km map file creation. Have
the number of steps that are skipped at startup dependent on the time-step size. Add
a test for some requirements of WACCMX (time-step and use of ESMF). Calculations of 
local time are now done in a global subroutine, that can handle negative longitudes.
Fix how FFIX_TO_SMINN is handled for history output. The namelist logical "calc_human_stress_indices"
changed from logical to a character string of three values: FAST, NONE, ALL. FAST
is the default so the less expensive ones are output, NONE turns them all off, and ]
ALL does all of them including the expensive ones.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): #428, #474, #475, #476, #450, #482, #481, #491
   Fix #428 -- Update getco2_historical.ncl to handle latitude varying CO2
   Fix #474 -- Add ability to send GRIDFILE to regridbatch.sh script
   Fix #475 -- Have number of steps to skip balance-check based on time
   Fix #476 -- Add a test for WACCMX standalone
   Fix #450 -- Add option to use global average of terrain standard deviation on surfdata files
       (partial fix with simplest option)
   Fix #482 -- Add extra field on CO2 streams file for global/time-averaged data
   Fix #481 -- FFIX_TO_SMINN needs to be output when FUN is on
   Fix #491 -- Calculations of local noon assume that longitude is 0 to 360 rather than -180 to 180

Known bugs introduced in this tag (include github issue ID): cime#2801
   cime#2801 -- Problem building with ESMF_LIB

Known bugs found since the previous tag (include github issue ID):  #507, #505
   #507 -- Albedo's are bad at night with negative longitudes
   #505 -- CTSM input data-set tools can only work on 0-360 grids, and require monotone increasing longitude


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

[ ] clm4_0

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): None

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables):

  New namelist:
    use_ssre -- Turn on show free fields needed for SnowMIP

  Changed namelist:
   calc_human_stress_indices changed from logical to character with options: ALL, FAST, NONE

  New history fields:
     Mostly added Snow Free (SF) fields
     ALBDSF
     ALBISF
     FSRSF
     FSRSFND
     FSRSFNDLN
     FSRSFNI
     FSRSFVD
     FSRSFVDLN
     FSRSFVI
     SSRE_FSR
     SSRE_FSRND
     SSRE_FSRNDLN
     SSRE_FSRNI
     SSRE_FSRVD
     SSRE_FSRVDLN
     SSRE_FSRVI

Changes made to namelist defaults (e.g., changed parameter values): 
   calc_human_stress_indices = 'FAST' is now the default

Changes to the datasets (e.g., parameter, surface or initial files):
   mkghg_bndtvghg -- Update with new CO2 files, both monthly, lat-bands and yearly, global

Substantial timing or memory changes: None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in .CTSMTrunkChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance):
   I was able to reduce the duplication in SurfaceAlbedoMod where the original implementation added a cut
   and paste copy of code. But, there is still a lot of duplication in this file that could be improved, by
   making smaller functions/subroutines to do sections of code that are essentially repeated many times.
   There's a bit of an increase in complexity to reduce the duplication, but reducing the duplication was worth it.

Changes to tests or testing:
   Add a new waccmx_offline test mods and test with it
   New test expected fail because of cime issue: ERS_D_Ln9_P480x3.f19_g16.I2000Clm50SpGs.cheyenne_intel.clm-waccmx_offline
   Turn use_ssre on for most tests, off for reducedOutput
   And set calc_human_stress_indices=NONE for reducedOutput, FAST for most, and ALL for KitchenSink and KSMOut tests

Code reviewed by: self, @olyson, @billsacks


CTSM testing: regular

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS

  unit-tests (components/clm/src):

    cheyenne - PASS

  regular tests (aux_clm):

    cheyenne_intel ---- OK
    cheyenne_gnu ------ OK
    hobart_nag -------- OK
    hobart_pgi -------- OK
    hobart_intel ------ OK

CTSM tag used for the baseline comparisons: ctsm1.0.dev011


Answer changes
--------------

Changes answers relative to baseline: no bit-for-bit

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): No

Pull Requests that document the changes (include PR ids): #462 #449
(https://github.com/ESCOMP/ctsm/pull)

   #462 -- Add namelist item to calculate all heat stress indices only if requested; to speed up model
   #449 -- Bring in snowmip diagnostic fields

===============================================================
===============================================================
Tag name:  ctsm1.0.dev011
Originator(s):  sacks (Bill Sacks), mvr (Mathew Rothstein)
Date: Wed Sep 12 10:50:31 MDT 2018
One-line Summary: Add water tracer consistency checks, and other water tracer work

Purpose of changes
------------------

1. Add water tracer consistency checks

2. Add infrastructure for looping over all water tracers - currently
   just used for the tracer consistency checks

3. Breakout of atm2lnd and lnd2atm water variables, needed for water tracers

4. Add some namelist control over the addition of water tracers

5. Add a system test that exercises the water tracer consistency checks

6. Add a 'ratio' variable for each water tracer

7. Add some unit tests of the new water tracer infrastructure

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Partially addresses #357
- Resolves #479
- Resolves #492

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

[ ] clm4_0

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions):
- Don't be fooled by the new namelist variable, enable_water_isotopes:
  This is just a place-holder for now, not implying that water isotopes
  are actually working.

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables):
- New namelist variables: enable_water_tracer_consistency_checks and
  enable_water_isotopes. The latter is just a place-holder for now.

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in .CTSMTrunkChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: New test that runs the water tracer consistency check
   I ran this test on cheyenne_gnu and cheyenne_intel along with the
   out-of-the-box hobart_nag version

Code reviewed by: Mat Rothstein and I have worked together on many of
these changes, but not all code has been reviewed by the other.

CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - ok (tests pass, clm4_5 and clm5_0 namelists differ from
    baseline as expected)

  unit-tests (components/clm/src):

    cheyenne - pass

  tools-tests (components/clm/test/tools):

    cheyenne - not run

  PTCLM testing (components/clm/tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne_intel ---- pass
    cheyenne_gnu ------ pass
    hobart_nag -------- pass
    hobart_pgi -------- pass
    hobart_intel ------ pass

CTSM tag used for the baseline comparisons: ctsm1.0.dev010


Answer changes
--------------

Changes answers relative to baseline: NO

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
- https://github.com/ESCOMP/ctsm/pull/497

===============================================================
===============================================================
Tag name:  ctsm1.0.dev010
Originator(s):  sacks (Bill Sacks)
Date: Thu Aug 30 17:14:28 MDT 2018
One-line Summary: Update cime to cime5.7.3

Purpose of changes
------------------

Update cime from cime5.6.10 to cime5.7.3. To support this change, there
are also minor code changes related to the pause-resume implementation
(from Erik Kluzek).

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- ESCOMP/ctsm#384 (VIC test is failing at f09 resolution with signal)
  (I'm not sure what fixed this, but it's passing now)

CIME Issues fixed (include issue #): many


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

[ ] clm4_0

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in .CTSMTrunkChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: self


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  unit-tests (components/clm/src):

    cheyenne - pass

  tools-tests (components/clm/test/tools):

    cheyenne - not run

  PTCLM testing (components/clm/tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne_intel ---- pass
    cheyenne_gnu ------ pass
    hobart_nag -------- pass
    hobart_pgi -------- pass
    hobart_intel ------ pass

CTSM tag used for the baseline comparisons: ctsm1.0.dev009


Answer changes
--------------

Changes answers relative to baseline: NO - bit-for-bit

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.):
cime: cime5.6.10 -> cime5.7.3

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev009
Originator(s):  sacks (Bill Sacks)
Date: Wed Aug 22 20:32:36 MDT 2018
One-line Summary: Fix initialization of AnnET in InitAccVars

Purpose of changes
------------------

InitAccVars was mistakenly setting qflx_evap_tot_col rather than
AnnET. This fix allows us to remove now-redundant cold start and restart
code for AnnET.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Fixes #480
- Partially addresses #285


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

[ ] clm4_0

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in .CTSMTrunkChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: basic proposed changes reviewed by Erik Kluzek


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  unit-tests (components/clm/src):

    cheyenne - pass

  tools-tests (components/clm/test/tools):

    cheyenne - not run

  PTCLM testing (components/clm/tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne_intel ---- pass
    cheyenne_gnu ------ pass
    hobart_nag -------- pass
    hobart_pgi -------- pass
    hobart_intel ------ pass

CTSM tag used for the baseline comparisons: ctsm1.0.dev008


Answer changes
--------------

Changes answers relative to baseline: NO

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids): none

===============================================================
===============================================================
Tag name:  ctsm1.0.dev008
Originator(s):  erik (Erik Kluzek)
Date: Tue Aug 14 10:25:12 MDT 2018
One-line Summary: Update 1850 ndep file and last year for streams for Historical transient cases

Purpose of changes
------------------

Bring in changes from release-clm5.0.05. Update to latest Nitrogen Deposition file from simulations with WACCM for 1850.
Also fix an issue with the last year for historical transient cases.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): 461
   #461 -- increase last year in streams for transient

Known bugs found since the previous tag (include github issue ID): [If none, remove this line]
  #478 -- Bare soil g1 should be missing value or zero


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

[ ] clm4_0

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): None

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): None

Changes made to namelist defaults (e.g., changed parameter values): Last year extended for transient datasets

Changes to the datasets (e.g., parameter, surface or initial files): New ndep dataset for 1850

Substantial timing or memory changes:

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in .CTSMTrunkChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): None

Changes to tests or testing: Lengthen some tests

Code reviewed by: self


CTSM testing: regular

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS (11 show differences for 1850_control and 20thC_transient)

  unit-tests (components/clm/src):

    cheyenne - PASS

  regular tests (aux_clm):

    cheyenne_intel ---- OK
    cheyenne_gnu ------ OK
    hobart_nag -------- OK
    hobart_pgi -------- OK
    hobart_intel ------ OK

CTSM tag used for the baseline comparisons: ctsm1.0.dev007


Answer changes
--------------

Changes answers relative to baseline: Yes!

  Summarize any changes to answers, i.e.,
    - what code configurations: 1850_control or 20thC_transient for Clm50
    - what platforms/compilers: all
    - nature of change: similar climate

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): None

Pull Requests that document the changes (include PR ids):
(https://github.com/ESCOMP/ctsm/pull)

    #477 -- Move changes from release-clm5.0.05 onto master

===============================================================
===============================================================
Tag name:  ctsm1.0.dev007
Originator(s):  sacks (Bill Sacks)
Date: Sun Aug  5 21:03:28 MDT 2018
One-line Summary: Avoid glacier dynamic landunit adjustments in first time step

Purpose of changes
------------------

Always avoid generating dynamic landunit adjustments for glacier area
changes in the first timestep of a startup or hybrid run - not just for
cold start or interpolated start. See the detailed discussion in
https://github.com/ESCOMP/ctsm/issues/340 for rationale.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Fixes #340 (Avoid generating dynamic landunit adjustment fluxes for
  glacier changes in the first timestep)


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

[ ] clm4_0

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in .CTSMTrunkChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing: none

Code reviewed by: Erik Kluzek


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  unit-tests (components/clm/src):

    cheyenne - not run

  tools-tests (components/clm/test/tools):

    cheyenne - not run

  PTCLM testing (components/clm/tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne_intel ---- ok
    cheyenne_gnu ------ pass
    hobart_nag -------- pass
    hobart_pgi -------- pass
    hobart_intel ------ pass

    ok means tests pass; baseline failures as expected in these two
    tests that do not use init_interp:

    FAIL ERI_N2_Ld9.f19_g17.I2000Clm50BgcCrop.cheyenne_intel.clm-default BASELINE ctsm1.0.dev006
    FAIL SMS_Lm13.f19_g17.I2000Clm50BgcCrop.cheyenne_intel.clm-cropMonthOutput BASELINE ctsm1.0.dev006

    (The glacier area on the finidat file used in these tests -
    clmi.I2000Clm50BgcCrop.2011-01-01.1.9x2.5_gx1v7_gl4_simyr2000_c180715.nc
    - seems to have come from an finidat_interp_dest file, and thus does
    not match the glacier area from CISM.)

CTSM tag used for the baseline comparisons: ctsm1.0.dev006


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: Startup / Hybrid runs that include CISM
     (even NOEVOLVE) and do not use init_interp, due to:
     - different PE layout in new case vs. the one that generated the
       finidat (roundoff-level diffs)
     - different glacier area on finidat file vs. what's in CISM
       (greater than roundoff-level diffs)
    - what platforms/compilers: all
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 
      Roundoff-level for different PE layout, larger than roundoff but
      should be same climate when there is different glacier area on the
      finidat file vs. what's in CISM

      The differences arise from dynamic landunit adjustments of the
      below-ground C and N states in the old code, as CTSM adjusts its
      areas to match CISM's. In the new code, there are no C and N
      adjustments in the first time step from this adjustment.

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
- #470 (Avoid glacier dynamic landunit adjustments in first time step)

===============================================================
===============================================================
Tag name:  ctsm1.0.dev006
Originator(s):  sacks (Bill Sacks)
Date: Sat Aug  4 07:48:09 MDT 2018
One-line Summary: Minor bug fixes, cleanup, documentation and enhancements

Purpose of changes
------------------

A collection of minor bug fixes, code cleanup, documentation and
enhancements, all bit-for-bit. See list of issues fixed below for
details.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Fixes #24 (ncd_io_1d_log_glob is broken)
- Fixes #120 (Incorrect comments in Biogeophysics1Mod.F90
- Fixes #217 (Change some cheyenne_gnu tests to cheyenne_intel)
- Fixes #245 (Put all .gitignore entries in top-level file)
- Fixes #272 (Code should error on missing mxsoil_color when SOIL_COLOR
  is used)
- Fixes #283 (Add more helpful message about need to do init_interp with
  wrong number of vertical layers)
- Fixes #367 (For cmip6 runs: Turn on cpl hist output needed to drive a
  TG compset)
- Fixes #412 (Fix documentation of init_interp_method)
- Fixes #419 (Do not allow SOYFIXN diagnostic field with FUN)
- Fixes #464 (Some lines longer than 132 characters)
- Fixes #465 (Remove backwards compatibility check for snw_rds)
- Fixes #467 (Increase wallclock limit for test)


Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

[ ] clm4_0

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in .CTSMTrunkChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): none

Changes to tests or testing:
- Changed a single-point test from gnu to intel to get better debugging
   information
   (SMS_D_Ly6_Mmpi-serial.1x1_smallvilleIA.IHistClm45BgcCropQianGs.cheyenne_intel.clm-cropMonthOutput)
- For some single-point tests on both gnu and intel: removed one
  compiler, keeping the tests on just one

Code reviewed by: self


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  unit-tests (components/clm/src):

    cheyenne - pass

  tools-tests (components/clm/test/tools):

    cheyenne - not run

  PTCLM testing (components/clm/tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne_intel ---- pass
    cheyenne_gnu ------ pass
    hobart_nag -------- pass
    hobart_pgi -------- pass
    hobart_intel ------ pass

    NLCOMP failures as expected for tests with the cmip6 test mod

    Some long single-point gnu tests failed due to running out of
    wallclock time; I have removed these, just keeping them on intel
    (along with removing a few other tests after I ran the testing - see
    commit 5c226bdf):

       FAIL ERS_Lm54_Mmpi-serial.1x1_numaIA.I2000Clm50BgcCropGs.cheyenne_gnu.clm-cropMonthOutput RUN time=4803
       FAIL ERS_Ly6_Mmpi-serial.1x1_smallvilleIA.IHistClm50BgcCropGs.cheyenne_gnu.clm-cropMonthOutput RUN time=6024


CTSM tag used for the baseline comparisons: ctsm1.0.dev005


Answer changes
--------------

Changes answers relative to baseline: NO - bit-for-bit
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
- #392 (Remove old code for snw_rds restart that looks no longer needed)
- (Other changes not documented in PRs)

===============================================================
===============================================================
Tag name:  ctsm1.0.dev005
Originator(s):  sacks (Bill Sacks), mvr (Mathew Rothstein)
Date: Fri Aug  3 07:54:59 MDT 2018
One-line Summary: Rework water data types to accommodate isotopes and other tracers

Purpose of changes
------------------

This tag reworks the various water data types to allow having multiple
instances of variables that are needed for isotopes and other water
tracers.

Specific changes include:

(1) Separated "water state" variables into state, diagnostic and balance
    check-related variables. This separation was not essential for the
    work here, but was desired by Martyn Clark and others.

(2) For each of water state, diagnostic and flux variables, separated
    variables into those needed for both bulk and tracers vs. those only
    needed for bulk. This way, we can have multiple instances of the
    variables needed by tracers, but only a single instance of variables
    that only apply to bulk water. This follows the design laid out in
    https://github.com/escomp/ctsm/pull/395. The separation was based
    largely on what was done in the old water isotope branch; we didn't
    put a lot of thought into this, because the new design allows us to
    easily migrate variables between bulk-only and bulk-and-tracer as
    needed.

(3) Moved water fluxes that were defined in science modules back into
    waterflux_type or waterfluxbulk_type. This was needed for (2); there
    is more discussion on this in
    https://github.com/escomp/ctsm/pull/395 and the log message for
    commit 711e5cd7.

(4) Introduced a top-level water_type that holds instances of all of the
    other water-related objects. This follows the design laid out in
    https://github.com/escomp/ctsm/pull/395. This is particularly
    valuable for the tracer instances: the logic related to number of
    tracers can be encapsulated in water_type, rather than infiltrating
    clm_instMod.

(5) Added placeholders for water tracer instances

(6) Added infrastructure to generate history / restart field names for
    the tracer instances. Eventually, the isotope class can also hold
    information specific to each isotope.

This work was a joint effort between Mathew Rothstein and myself; Mat
gets much of the credit for the actual refactoring done here.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Fixes #358 (Separate WaterStateType into multiple types)
- Fixes #434 (Separate WaterFluxType into a base class and a class that just applies to bulk)
- Fixes #359 (Set up infrastructure for multiple instances of WaterState and WaterFlux types)
- Fixes #458 (Implement handling of history and restart variables for water tracers)

Known bugs introduced in this tag (include github issue ID):
- #464 (Some lines longer than 132 characters)
- #456 (Remove backwards compatibility check for snw_rds) (not a
  newly-introduced issue, but it's more important to fix this with the
  separation of waterstate_type)

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

[ ] clm4_0

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): none

Changes made to namelist defaults (e.g., changed parameter values): none

Changes to the datasets (e.g., parameter, surface or initial files): none

Substantial timing or memory changes: none

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in .CTSMTrunkChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance):
- Water tracers are not yet functional - they are just placeholders
- Whether to use water tracers and which water tracers to use are
  currently hard-coded; eventually, this should be namelist-controlled

Changes to tests or testing: none

Code reviewed by: Mathew Rothstein and Bill Sacks worked together on
these changes and reviewed each other's changes to some extent. The
high-level design was also reviewed and co-developed by Mariana
Vertenstein. Martyn Clark contributed substantially to the breakout of
water state into state, diagnostic and balance check-related variables.


CTSM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  unit-tests (components/clm/src):

    cheyenne - pass

  tools-tests (components/clm/test/tools):

    cheyenne - not run

  PTCLM testing (components/clm/tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne_intel ---- pass
    cheyenne_gnu ------ pass
    hobart_nag -------- pass
    hobart_pgi -------- pass
    hobart_intel ------ pass

CTSM tag used for the baseline comparisons: ctsm1.0.dev004


Answer changes
--------------

Changes answers relative to baseline: NO - bit-for-bit


Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
- https://github.com/escomp/ctsm/pull/395 (documents the high-level design)

===============================================================
===============================================================
Tag name:  ctsm1.0.dev004
Originator(s):  erik (Erik Kluzek,UCAR/TSS,303-497-1326)
Date: Wed Jul 18 02:10:23 MDT 2018
One-line Summary: Add some new diagnostic fields, fix a few issues, update cmip6 output

Purpose of changes
------------------

Add some new diagnostic fields. Some needed for CMIP6. Update the CMIP6 user-mods output.
Fix a couple issues. Get full list of history tapes working correctly. Check for valid range
of CO2. New IC file interpolated from the previous one for f19_g17_gl4 for 2000 Clm50BgcCrop.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): $210, #427, #429
   #210 -- increase number of history tapes
   $427 -- Check for zero CO2
   #429 -- New IC for present day

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations? No
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

[ ] clm4_0

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): none

Changes to CTSM's user interface (e.g., new/renamed XML or namelist variables): New history fields
   TSKIN --- skin temperature
   TSL ----- temperature of near-surface soil layer
   GSSUNLN - sunlit leaf stomatal conductance at local noon
   GSSHALN - shaded leaf stomatal conductance at local noon

Changes made to namelist defaults (e.g., changed parameter values):  None

Changes to the datasets (e.g., parameter, surface or initial files): New finidat file f19_g17_gl4 2000 Clm50BgcCrop 
    New file is just interpolate from the previous file to a f19_g17_gl4 gridfor Clm50BgcCrop and 2000 simulation year

Substantial timing or memory changes: None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in .CLMTrunkChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance):

Changes to tests or testing: None

Code reviewed by: self, oleson, abtawfik


CTSM testing: regular

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - OK (95 are different because of finidat file change)

  unit-tests (components/clm/src):

    cheyenne - PASS

  regular tests (aux_clm):

    cheyenne_intel ---- OK
    cheyenne_gnu ------ OK
    hobart_nag -------- PASS
    hobart_pgi -------- FAIL
    hobart_intel ------ PASS

CTSM tag used for the baseline comparisons:  ctsm1.0.dev003


Answer changes
--------------

Changes answers relative to baseline: Yes -- only 2000 cases because of new IC

  Summarize any changes to answers, i.e.,
    - what code configurations: Just 2000 cases
    - what platforms/compilers: All
    - nature of change: same climate

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): None

Pull Requests that document the changes (include PR ids):
(https://github.com/ESCOMP/ctsm/pull)


   #338 -- Radtemp and other new diagnostic fields

===============================================================
===============================================================
Tag name:  ctsm1.0.dev003
Originator(s):  erik (Erik Kluzek,UCAR/TSS,303-497-1326)
Date: Sun Jul 15 00:26:46 MDT 2018
One-line Summary: Update cime/cism to work on upgraded hobart and with glade changes on cheyenne

Purpose of changes
------------------

Update cime and cism so can work on upgraded hobart. Also fix an issue that the Nag6.2 compiler found
in CTSM. Also get working with glade changes that happened on cheyenne, DIN_LOC_ROOT_CLMFORC was
changed to a new directory.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): [If none, remove this line]
   #441 -- Changes in glade invalidate previous softlinks and data locations
   #438 -- Illegal argument aliasing caught by nag6.2
   #433 -- with hobart upgrade CTSM not working

Known bugs introduced in this tag (include github issue ID):
   #444 -- PGI fails on cheyenne and hobart

Known bugs found since the previous tag (include github issue ID):
   #443 -- Wood C:N ratios
   #440 -- leafcn_max < target leafcn?

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations? No
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm5_0

[ ] clm4_5

[ ] clm4_0

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): None

Changes to CLM's user interface (e.g., new/renamed XML or namelist variables): DIN_LOC_ROOT_CLMFORC on cheyenne
   DIN_LOC_ROOT_CLMFORC was change on cheyenne

Changes made to namelist defaults (e.g., changed parameter values): None

Changes to the datasets (e.g., parameter, surface or initial files): None

Substantial timing or memory changes: None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in .CTSMTrunkChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): None

Changes to tests or testing: path for cheyenne specific test had to be changed
    Add some gnu tests on hobart
    Lengthen wallclock on a test
    Fix path for DA test_mod

Code reviewed by: self

CLM testing: regular

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS

  unit-tests (components/clm/src):

    cheyenne - PASS

  tools-tests (components/clm/test/tools):

    cheyenne - PASS

  PTCLM testing (components/clm/tools/shared/PTCLM/test):

     cheyenne - OK

  regular tests (aux_clm):

    cheyenne_intel ---- OK
    cheyenne_gnu ------ OK
    hobart_nag -------- OK
    hobart_pgi -------- FAIL
    hobart_gnu -------- PASS
    hobart_intel ------ PASS

CLM tag used for the baseline comparisons:  ctsm1.0.dev002


Answer changes
--------------

Changes answers relative to baseline: No bit-for-bit

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): cime, cism
   cism-wrapper to release-cesm2.0.04
   cime         to cime5.6.10

Pull Requests that document the changes (include PR ids):
(https://github.com/ESCOMP/ctsm/pull)
 #444 -- Get working on upgraded hobart and with glade changes on cheyenne

===============================================================
===============================================================
Tag name:  ctsm1.0.dev002
Originator(s):  erik (Erik Kluzek)
Date: Fri Jul  6 16:33:03 MDT 2018
One-line Summary: Fix NFIX flux variables so special land-units are zeroed out, tools update, add some *_MAX fields on mksurfdata_map for transient cases

Purpose of changes
------------------

Nitrogen Fixation flux arrays were being set to missing value over non-vegetated landunits. This sets them to zero everywhere
and averages in the zero's at the gridcell level for history output. It also reads in restarts with missing values and converts
them to zero's.

There are also some tools updates, getting the tools working on the new upgraded hobart.cgd.ucar.edu.

And mksurfdata_map is updated to add some *_MAX files on the landuse.timeseries files that will allow us to conserve memory
for transient cases.


Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #): #435, #436, #433 (for tools)
    #435 -- intel build for tools
    #426 -- Nitrogen Fixation flux variables
    #433 -- hobart broken, got it working for tools

Known bugs found since the previous tag (include github issue ID): #433, #431

   #433 -- with hobart upgrade, hobart build isn't working
   #431 -- When set_* options are used to hist_addfld be careful of ordering

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

    [Put an [X] in the box for any configuration with significant answer changes.]

[X] clm5_0

[ ] clm4_5

[ ] clm4_0

Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions): None

Changes to CLM's user interface (e.g., new/renamed XML or namelist variables): None

Changes made to namelist defaults (e.g., changed parameter values): ndep file updated for clm5_0 for 1850

Changes to the datasets (e.g., parameter, surface or initial files): None

Substantial timing or memory changes: None

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in .CLMTrunkChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance): NFIX flux variables
   NFIX flux variables when read from restart have special-values (missing on non veg landunits) converted to zero's
   They are now initialized to zero everywhere.

Changes to tests or testing: None

Code reviewed by: self, wwieder, and lawrence, sacks (mksurfdata_map code)

CLM testing: regular (although none on hobart!)

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - PASS (5 tests differ from previous version)

  unit-tests (components/clm/src):

    cheyenne - PASS

  tools-tests (components/clm/test/tools):

    cheyenne - PASS
    hobart --- PASS

  regular tests (aux_clm):

    cheyenne_intel ---- OK
    cheyenne_gnu ------ OK

CLM tag used for the baseline comparisons: ctsm1.0.dev001


Answer changes
--------------

Changes answers relative to baseline: clm5_0 BGC configurations with FUN and clm5_0 with BGC for new 1850 ndep file

  Summarize any changes to answers, i.e.,
    - what code configurations: All clm5_0 BGC due to NFIX fields, and clm5_0 1850 BGC due to new ndep file
    - what platforms/compilers: All
    - nature of change: similar climate

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): None

Pull Requests that document the changes (include PR ids):
(https://github.com/ESCOMP/ctsm/pull)

  #437 -- ChangeLog template
  #432 -- nfix
  #331 -- _MAX fields on mksurfdata_map

===============================================================
===============================================================
Tag name:  ctsm1.0.dev001
Originator(s):  sacks (Bill Sacks)
Date: Fri Jun 22 07:31:43 MDT 2018
One-line Summary: Begin separating SoilHydrology flux calculations

Note the new tag naming: Starting with this tag, we are naming tags
"ctsm..." rather than "clm...". We are starting with ctsm version 1,
which for now is nearly the same as clm version 5. We are moving to this
new tag naming now because:

(1) The changes in this tag represent the first step towards
    implementing the CTSM vision (separating the biogeophysics flux
    calculations from state updates, making it easier to plug in
    alternative parameterizations, etc.).

(2) This tag changes answers relative to the CLM5 release. (We expect
    the climate to be the same, but we haven't tested this carefully
    yet.)

Purpose of changes
------------------

First steps toward separating various flux calculations in the soil
hydrology code. The focus here is on saturated surface runoff and
infiltration excess runoff. The changes here separate flux calculations
from state updates and extract various calculations into their own
subroutines to facilitate swapping in alternative parameterizations.

Most of the changes here are refactorings that are either bit-for-bit or
just introduce roundoff-level differences. However, there are also a few
larger answer changes, as described below.

These are the greater-than-roundoff-level answer changes:

(A1) Use full qflx_surf in BGC code. Previously, the subroutines ch4 and
     SoilBiogeochemNLeaching had been using a flux that excluded surface
     water runoff (qflx_h2osfc_surf). That was deemed to be incorrect
     (by Dave Lawrence and others), so these BGC subroutines have been
     changed to use the full surface runoff (saturated excess runoff +
     infiltration excess runoff + h2osfc runoff).

     Configurations affected: BGC compsets

     Magnitude of change: Larger than roundoff; expected to be same
     climate, but not investigated carefully

(A2) VIC: Don't ever use the TOPModel formulation for SurfaceRunoff

     The code was using the TOPModel-based formulation for SurfaceRunoff
     if frost_table > zwt_perched. Sean Swenson felt this shouldn't be
     done, and refactoring will be easier if VIC always uses the
     vic-looking formulation, so I'm stopping applying this formulation
     when using VIC.

     Configurations affected: VIC compsets

     Magnitude of change: Larger than roundoff; not investigated carefully

(A3) VIC: Remove infiltration excess runoff

     Martyn Clark reviewed the VIC implementation, and felt that the
     current implementation of infiltration excess runoff is
     inconsistent with the standard VIC implementation. It appears that
     what was being called VIC's infiltration excess runoff was actually
     just an attempt to give a better numerical approximation to the
     solution for saturated surface excess runoff. So deleting this
     leaves only a first-order approximation to VIC's saturated surface
     excess runoff.

     Eventually we may want to put in place a more accurate solution for
     VIC's saturated surface excess runoff. But Martyn's feeling is that
     this can come in with other changes we want to make regarding
     numerical solutions in CTSM.

     Configurations affected: VIC compsets

     Magnitude of change: Larger than roundoff; expected to be same
     climate, but not investigated carefully

(A4) Change in QOVER diagnostic field: now includes QH2OSFC.

These are the major refactorings (either bit-for-bit or just
roundoff-level differences):

(R1) Extract surface runoff to its own module, and other modularization
     related to what used to be subroutine SurfaceRunoff: extract a
     subroutine for each fsat method, move urban surface runoff code
     into different routines.

(R2) Extract infiltration excess runoff to its own module, and other
     modularization related to what used to be subroutine Infiltration.

NOTE: For a detailed breakdown of changes, including documentation of
which changed changed answers, see the file ChangeLog_branch which was
deleted from this branch shortly before it was merged to master.

Bugs fixed or introduced
------------------------

Issues fixed (include CTSM Issue #):
- Fixes #424 (decStart testmod is missing an `--append` in an xmlchange command)


Notes of particular relevance for users
---------------------------------------

Caveats for users (e.g., need to interpolate initial conditions):

Changes to CLM's user interface (e.g., new/renamed XML or namelist variables):

Changes made to namelist defaults (e.g., changed parameter values):

Changes to the datasets (e.g., parameter, surface or initial files):

Substantial timing or memory changes:

Notes of particular relevance for developers: (including Code reviews and testing)
---------------------------------------------
NOTE: Be sure to review the steps in .CLMTrunkChecklist as well as the coding style in the Developers Guide

Caveats for developers (e.g., code that is duplicated that requires double maintenance):

Changes to tests or testing:
- Shortened a 25-month f19 test to 13-months
  (SMS_Lm13.f19_g17.I2000Clm50BgcCrop.cheyenne_intel.clm-cropMonthOutput):
  This test took a long time to run, and we already have long tests of
  this configuration at coarse-resolution. I suggested removing this
  test entirely, but Erik Kluzek wanted to maintain a shortened version
  of it. Erik's comments were:

     How about either flip it to f45 (because Rosie is using that
     resolution for science) -- or leave it at f19 and have it run for 9
     months? The 25 months was to take it just past the 2-year spinup
     time. From spunup initial conditions 9-months pretty much does a
     full harvest cycle. f19 and f09 are the main configurations for
     science and it would be good to have something longer than just
     really short tests.

  I decided on Lm13, because this still completes in significantly less
  than 2-hours, and if we're running for 9 months anyway, it seems
  valuable to finish out the year and run a bit into the next year,
  since some crop stuff triggers at the end of the year.

Code reviewed by: Various code reviews done last Fall, when these
changes were first implemented.

CLM testing:

 [PASS means all tests PASS and OK means tests PASS other than expected fails.]

  build-namelist tests:

    cheyenne - not run

  unit-tests (components/clm/src):

    cheyenne - PASS

  tools-tests (components/clm/test/tools):

    cheyenne - not run

  PTCLM testing (components/clm/tools/shared/PTCLM/test):

     cheyenne - not run

  regular tests (aux_clm):

    cheyenne_intel ---- OK
    cheyenne_gnu ------ OK
    hobart_nag -------- OK
    hobart_pgi -------- OK
    hobart_intel ------ OK

    OK means tests pass, answers change as expected.

CLM tag used for the baseline comparisons: clm5.0.dev013


Answer changes
--------------

Changes answers relative to baseline: YES

  If a tag changes answers relative to baseline comparison the
  following should be filled in (otherwise remove this section):

  Summarize any changes to answers, i.e.,
    - what code configurations: Essentially all (except CLM4)
    - what platforms/compilers: All
    - nature of change (roundoff; larger than roundoff/same climate; new climate): 

    Roundoff-level changes for nearly all configurations (except CLM4).

    Greater-than-roundoff-level changes for the following
    configurations:
    - BGC: see (A1) note above (expected to be same climate, but not
      investigated carefully)
    - VIC: see (A2) and (A3) notes above (magnitude of change not
      investigated carefully)
    - QOVER diagnostic field: see (A4) note above

   If bitwise differences were observed, how did you show they were no worse
   than roundoff? N/A

   If this tag changes climate describe the run(s) done to evaluate the new
   climate (put details of the simulations in the experiment database)
       - casename: N/A

   URL for LMWG diagnostics output used to validate new climate: N/A
	

Detailed list of changes
------------------------

List any externals directories updated (cime, rtm, mosart, cism, fates, etc.): none

Pull Requests that document the changes (include PR ids):
- https://github.com/ESCOMP/ctsm/pull/244
- https://github.com/ESCOMP/ctsm/pull/190
- Other PRs that existed on the old NCAR/clm-ctsm repository (which has
  been deleted), which were not migrated to the new repository

===============================================================
