diff --git a/docs/INDEX.md b/docs/INDEX.md index 76c7c62cff..24e456236c 100644 --- a/docs/INDEX.md +++ b/docs/INDEX.md @@ -1,8 +1,49 @@ -# eCLM Documentation +# Welcome to eCLM! -```{important} -**Welcome!** You are viewing the first version of the documentation for eCLM. This is a living document, which means it will be continuously updated and improved. Please check back regularly for the latest information and updates. +eCLM is based on version 5.0 of the Community Land Model ([CLM5](https://www.cesm.ucar.edu/models/clm)) The Community Land Model is the land model for the Community Earth System Model (CESM). It focuses on modeling the land as the critical interface through which humanity affects, adapts to, and mitigates global environmental change. It includes a comprehensive representation of land biogeophysics, hydrology, plant physiology, biogeochemistry, anthropogenic land use, and ecosystem dynamics across a variety of spatial and temporal scales. The central theme is that terrestrial ecosystems, through their cycling of energy, water, chemical elements, and trace gases, are important determinants of climate. +The model represents several aspects of the land surface including surface heterogeneity and consists of components or submodels related to land biogeophysics, the hydrologic cycle, biogeochemistry, human dimensions, and ecosystem dynamics. Specific processes that are represented include: + +- Vegetation composition, structure, and phenology +- Absorption, reflection, and transmittance of solar radiation +- Absorption and emission of longwave radiation +- Momentum, sensible heat (ground and canopy), and latent heat (ground evaporation, canopy evaporation, transpiration) fluxes +- Heat transfer in soil and snow including phase change +- Canopy hydrology (interception, throughfall, and drip) +- Snow hydrology (snow accumulation and melt, compaction, water transfer between snow layers) +- Soil hydrology (surface runoff, infiltration, redistribution of water within the column, sub-surface drainage, groundwater) +- Plant hydrodynamics +- Stomatal physiology and photosynthesis +- Lake temperatures and fluxes +- Dust deposition and fluxes +- Routing of runoff from rivers to ocean +- Volatile organic compounds emissions +- Urban energy balance and climate +- Carbon-nitrogen cycling +- Dynamic landcover change +- Land management including crops and crop management and wood harvest +- Ecosystem Demography (FATES, optional) + +```{figure} users_guide/images/CLM5_processes_Lawrence2019.png +:height: 400px +:name: fig1 + +Overview of land biogeophysical, biogeochemical and landscape processes simulated by CLM5 Lawrence et al. (2019). ``` -eCLM is based on version 5.0 of the Community Land Model ([CLM5](https://www.cesm.ucar.edu/models/clm)) with simplified infrastructure for build and namelist generation. The build system is handled entirely by Cmake and namelists are generated through a small set of Python scripts. Similar to CLM5, eCLM is forced with meteorological data and uses numerous input streams on soil properties, land cover and land use, as well as complex parameter sets on crop phenology, and plant hydraulics for simulations. +Each time step the model solves the surface energy balance, water balance, and carbon exchange. Submodels of CLM include biophysics and biogeochemistry. It represents land surface heterogeneity through a subgrid tiling structure and different plant functional types that have unique parameters in terms of optical properties, morphology, photosynthesis. It can be run with prescribed vegetation states using satelllite phenology (SP mode) or prognostic vegetation states and biogeochemistry (BGC mode). + +```{figure} users_guide/images/CLM5_subgrid_structure_v1.png +:height: 400px +:name: fig2 + +Configuration of the CLM subgrid hierarchy. Box in upper right shows hypothetical subgrid distribution for a single grid cell. Note that the Crop land unit is only used when the model is run with the crop model active. Abbreviations: TBD – Tall Building District; HD – High Density; MD – Medium Density, G – Glacier, L – Lake, U – Urban, C – Crop, V – Vegetated, PFT – Plant Functional Type, Irr – Irrigated, UIrr – Unirrigated. Red arrows indicate allowed land unit transitions. Purple arrows indicate allowed patch-level transitions. From CLM5 Technical Note. +``` + +More background information on CLM (and it’s latest release CLM5) including documentation and publications can be found here. + +## Difference with CLM5 + +eCLM has simplified infrastructure for build and namelist generation. The build system is handled entirely by Cmake and namelists are generated through a small set of Python scripts. Similar to CLM5, eCLM is forced with meteorological data and uses numerous input streams on soil properties, land cover and land use, as well as complex parameter sets on crop phenology, and plant hydraulics for simulations. + + diff --git a/docs/_toc.yml b/docs/_toc.yml index 7dd640808d..26be54042d 100644 --- a/docs/_toc.yml +++ b/docs/_toc.yml @@ -5,39 +5,25 @@ root: INDEX parts: - caption: Introduction chapters: - - file: users_guide/installation/README - title: Installing eCLM - - file: users_guide/introduction/introduction - title: Scientific Background - - file: users_guide/introduction/perturbation - title: Atmospheric forcing noise - - file: users_guide/introduction/soil_hydraulic_parameters - title: Soil hydraulic properties from surface file - + - file: users_guide/introduction/quick_start + title: Quick Start - caption: User's Guide chapters: - file: users_guide/case_examples/README title: Running example cases sections: - file: users_guide/case_examples/Wuestebach - - file: users_guide/case_examples/NRW - - file: users_guide/case_examples/EURO-CORDEX - - - file: users_guide/running_cases/case_customization - title: Customizing eCLM namelists - - file: users_guide/analyzing_model_output - title: Analyzing model output - - - file: users_guide/case_creation/README + - url: https://hpscterrsys.github.io/eCLM_static-file-generator/INDEX.html title: Creating a custom case + + - file: users_guide/misc_tutorials/README + title: Miscellaneous tutorials sections: - - file: users_guide/case_creation/1_create_grid_file - - file: users_guide/case_creation/2_create_mapping_file - - file: users_guide/case_creation/3_create_domain_file - - file: users_guide/case_creation/4_create_surface_file - - file: users_guide/case_creation/5_modifications_surface_domain_file - - file: users_guide/case_creation/6_create_atm_forcings + - file: users_guide/misc_tutorials/perturbation + title: Atmospheric forcing noise + - file: users_guide/misc_tutorials/soil_hydraulic_parameters + title: Soil hydraulic properties from surface file - caption: Developer's Guide chapters: @@ -51,7 +37,3 @@ parts: - file: reference/history_fields - url: https://escomp.github.io/CTSM/release-clm5.0/tech_note/index.html title: CLM5 Technical Note - - url: https://github.com/HPSCTerrSys/eCLM_static-file-generator/blob/main/README.md) - title: eCLM static file generator - - url: https://hpscterrsys.github.io/TSMP2_workflow-engine - title: TSMP2 Workflow Engine diff --git a/docs/users_guide/analyzing_model_output.md b/docs/users_guide/analyzing_model_output.md deleted file mode 100644 index 3895ef7d22..0000000000 --- a/docs/users_guide/analyzing_model_output.md +++ /dev/null @@ -1,52 +0,0 @@ -# Analyzing model output - -As a reminder, the model output can be found in your case directory. All eCLM output files are in netCDF format (.nc). If you are running for different time sampling frequencies (set in the `lnd_in` namelist), there will be separate files created for each frequency, e.g. `my_case.h0.1993-01.nc` and `my_case.h1.1993-01.nc`. - -NetCDF is supported by many languages including Python, Matlab, R, NCL, IDL; tool suites of file operators (NCO, CDO) and viewing tools like ncview or ncdump. ncdump helps you to get an overview of the file structure and ncview is a graphical interface that allows to interactively visualize a selected variable across a selected range (time, spatial). - -## Using Ncview - -To use ncview you first have to load the ncview module. You also need Xming to be active to display the ncview window. Then you can open ncview and look at your file. Afterward, you can simply close the ncview interface to get back to your terminal - -```sh -module load ncview -ncview FILENAME.nc # replace FILENAME with your output history file -``` - -```{figure} images/ncview.png -:height: 400px -:name: fig6 - -Example of ncview interface. -``` - -## Using ncdump - -The utility ncdump can be used to show the contents of netCDF files. - -```sh -# Output a summary of your file (dimensions, variables, global attributes) -ncdump -h FILENAME.nc # replace FILENAME with your output history file - -# Show the data for a specific variable -ncdump -v VAR1 FILENAME.nc # replace VAR1 with the variable of interest and FILENAME with your output history file -``` - -See `ncdump --help` for more options. - -## Using Python - -For scientific output analysis and visualizations, programming languages like Python are recommended. - -You can use Jupyter Notebooks on JSC machines to directly access the model input and output data (login to Jupyter-JSC). -If you have access to Gitlab, you can check the tutorial for Jupyter Notebooks on JSC machines through the following link: JupyterLab on JSC. - -An exemplary notebook for eCLM output analysis and an annual output file covering the period from 2017-01-01 to 2017-12-31 for the NRW domain are available in the shared data directory `/p/scratch/cslts/shared_data/rlmod_eCLM/`. Copy the Jupyter Notebook to your directory before modifying it. - -```sh -cp /p/scratch/cslts/shared_data/rlmod_eCLM/Analyse_ECLM_Input_Output.ipynb $MYPROJECT -``` - -```{important} -Remember to stop the JupyterLab after usage! See point 3 in JupyterLab on JSC -``` \ No newline at end of file diff --git a/docs/users_guide/case_creation/1_create_grid_file.md b/docs/users_guide/case_creation/1_create_grid_file.md deleted file mode 100644 index 1a859968c7..0000000000 --- a/docs/users_guide/case_creation/1_create_grid_file.md +++ /dev/null @@ -1,55 +0,0 @@ -# Create SCRIP grid file - -The first step in creating your input data is to define your model domain and the grid resolution you want to model in. There are several options to create the SCRIP grid file that holds this information: -1. Using the `mkscripgrid.py` script to create a regular latitude longitude grid. -2. Using the `produce_scrip_from_griddata.ncl` script to convert an existing netCDF file that holds the latidude and longitude centers of your grid in 2D (This allows you to create a curvilinear grid). -3. Similar to the first option but using the `scrip_mesh.py` script to create the SCRIP grid file. - -To start the SCRIP grid file creation navigate into the `mkmapgrids` directory where you will find the above mentioned scripts. - -```sh -cd mkmapgrids -``` - -## 1. Create SCRIP grid file with `mkscripgrid.py` - -To use `mkscripgrid.py`, first open the script (for example using vim text editor) and adapt the variables that describe your grid. These include your grid name, the four corner points of your model domain as well as the resolution (lines 42-50 of the script). Then you can execute the script: - -```sh -python mkscripgrid.py -``` - -```{attention} -The `mkscripgrid.py` script requires numpy and netCDF4 python libraries to be installed (use pip install to do that if not already installed). -``` - -The output will be a SCRIP grid netCDF file containing the grid dimension and the center and corners for each grid point. It will have the format `SCRIPgrid_"Your grid name"_nomask_c"yymmdd".nc` - -## 2. Create SCRIP grid file from griddata with `produce_scrip_from_griddata.ncl` - -Unfortunately, NCL is not maintained anymore in the new software stages. Therefore, in order to use it you first need to load an older Stage and the required software modules: - -```sh -module load Stages/2020 -module load Intel/2020.2.254-GCC-9.3.0 -module load ParaStationMPI/5.4.7-1 -module load NCL -``` - -Next, adapt the input in `produce_scrip_from_griddata.ncl` to your gridfile.This includes choosing a name for your output file "OutFileName", adjusting the filename of your netcdf file in line 9 and the variable names for longitude/latitude in lines 10-11. Then execute: - -```sh -ncl produce_scrip_from_griddata.ncl -``` - -## 3. Create SCRIP grid file from griddata using `scrip_mesh.py` - -Alternatively to the first option, you can use the python script `scrip_mesh.py`. Like the ncl script it can create SCRIP files including the calculation of corners. It takes command line arguments like this: - -```sh -python3 scrip_mesh.py --ifile NC_FILE.nc --ofile OUTPUT_SCRIP.nc --oformat SCRIP # replace NC_FILE.nc with your netcdf file and choose a name for your output SCRIP grid file for OUTPUT_SCRIP.nc -``` - ---- - -**Congratulations!** You successfully created your SCRIP grid file and can now move on to the next step. \ No newline at end of file diff --git a/docs/users_guide/case_creation/2_create_mapping_file.md b/docs/users_guide/case_creation/2_create_mapping_file.md deleted file mode 100644 index 1e23c821a8..0000000000 --- a/docs/users_guide/case_creation/2_create_mapping_file.md +++ /dev/null @@ -1,39 +0,0 @@ -# Create mapping files - -To start the mapping file creation navigate into the `mkmapdata` directory where you will find the script needed for this step. - -```sh -cd ../mkmapdata -``` - -Before you run `runscript_mkmapdata.sh` you need to adapt some environment variables in lines 23-25 of the script. For this open the script (for example using vim text editor) and enter the name of your grid under `GRIDNAME` (same as what you used for the SCRIP grid file). For `CDATE`, use the date that your SCRIP grid file was created (per default the script uses the current date, if you created the SCRIPgrid file at some other point, you find the date of creation at the end of your SCRIPgrid file or in the file information). Lastly, provide the full path and name of your SCRIP grid file under `GRIDFILE`. Save and close the script. - -To create your mapping files, you need a set of rawdata. If you are a JSC user you can simply refer to the common data repository by adapting the "rawpath" path in line 29 of the script. - -``` -rawpath="/p/scratch/cslts/shared_data/rlmod_eCLM/inputdata/surfdata/lnd/clm2/mappingdata/grids" -``` - -For non JSC users, download the data and adapt "rawpath" to their new location. To download the data to the directory use: - -```sh -wget --no-check-certificate -i clm_mappingfiles.txt -``` - -Now you can execute the script: - -```sh -sbatch runscript_mkmapdata.sh -``` - -The output will be a `map_*.nc` file for each of the rawdata files. These files are the input for the surface parameter creation weighted to your grid specifications. - -To generate the domain file in the next step a mapfile is needed. This can be any of the generated `map_*.nc` files. So, set the environment variable `MAPFILE` for later use: - -```sh -export MAPFILE="path to your mapfiles"/"name of one of your map files" -``` - ---- - -**Congratulations!** You successfully created your mapping files and can now move on to the next step. \ No newline at end of file diff --git a/docs/users_guide/case_creation/3_create_domain_file.md b/docs/users_guide/case_creation/3_create_domain_file.md deleted file mode 100644 index 9ea49b9b2c..0000000000 --- a/docs/users_guide/case_creation/3_create_domain_file.md +++ /dev/null @@ -1,37 +0,0 @@ -# Create domain file - -In this step you will create the domain file for your case using `gen_domain`. First, you need to navigate into the `gen_domain_files/src/` directory and compile it with the loaded modules ifort, imkl, netCDF and netCDF-Fortran. - -```sh -cd ../gen_domain_files/src/ - -# Compile the script -ifort -o ../gen_domain gen_domain.F90 -mkl -lnetcdff -lnetcdf -``` -```{attention} -If you get a message saying "ifort: command line remark #10412: option '-mkl' is deprecated and will be removed in a future release. Please use the replacement option '-qmkl'" or the compiling fails, replace `-mkl` with `-qmkl`. -``` - -Before running the script you need to export the environment variable `GRIDNAME` (same as what you used for the SCRIP grid file and in the `runscript_mkmapdata.sh` script). - -```sh -export GRIDNAME="your gridname" -``` -Then you can run the script: -```sh -cd ../ -./gen_domain -m $MAPFILE -o $GRIDNAME -l $GRIDNAME -``` - -The output of this will be two netCDF files `domain.lnd.*.nc` and `domain.ocn.*.nc` that define the land and ocean mask respectively. The land mask will inform the atmosphere and land inputs of eCLM when running a case. - -However, `gen_domain` defaults the use of the variables `mask` and `frac` on these files to be for ocean models, i.e. 0 for land and 1 for ocean. So to use them you have to either manipulate the `domain.lnd.*.nc` file to have mask and frac set to 1 instead of 0 (WARNING: some netCDF script languages have `mask` as a reserved keyword e.g. NCO, use single quotation marks as workaround). -Or simply swap/rename the `domain.lnd.*.nc` and `domain.ocn.*.nc` file: - -```sh -mv domain.lnd."your gridname"_"your gridname"."yymmdd".nc temp.nc -mv domain.ocn."your gridname"_"your gridname"."yymmdd".nc domain.lnd."your gridname"_"your gridname"."yymmdd".nc -mv temp.nc domain.ocn."your gridname"_"your gridname"."yymmdd".nc -``` - -**Congratulations!** You successfully created your domain files and can now move on to the final next step to create your surface data. diff --git a/docs/users_guide/case_creation/4_create_surface_file.md b/docs/users_guide/case_creation/4_create_surface_file.md deleted file mode 100644 index fe1d654c70..0000000000 --- a/docs/users_guide/case_creation/4_create_surface_file.md +++ /dev/null @@ -1,43 +0,0 @@ -# Create surface file - -In this step you will create the surface data file using the `mksurfdata.pl` script. -First, we will compile the script with `make` in the `mksurfdata/src` directory. - - -```sh -cd ../mksurfdata/src - -# Compile the script -make -``` - -The script needs a few environment variables such as `GRIDNAME` (exported in the previous step), `CDATE` (date of creation of the mapping files which can be found at the end of each `map_*` file before the file extension) and `CSMDATA` (the path where the raw data for the surface file creation is stored) before executing the script. - -```sh -export CDATE=`date +%y%m%d` -export CSMDATA="/p/scratch/cslts/shared_data/rlmod_eCLM/inputdata/" # this works for JSC users only, for non JSC users see below - -# generate surfdata -./mksurfdata.pl -r usrspec -usr_gname $GRIDNAME -usr_gdate $CDATE -l $CSMDATA -allownofile -y 2000 -crop -``` - -```{tip} -The `-crop` option used in `./mksurfdata.pl` will create a surface file for BGC mode with all crops active. If you want to use SP mode, you should run without this option. - -Use `./mksurfdata.pl -help` to display all options possible for this script. -For example: -- `hirespft` - If you want to use the high-resolution pft dataset rather than the default lower resolution dataset (low resolution is at half-degree, high resolution at 3minute), hires is only available for present-day (2000) -``` - -**For non JSC users**: -Non JSC users can download the raw data from HSC datapub using this link or from the official rawdata repository using `wget` before submitting the script. - -```sh -wget "RAWDATA_LINK"/"NAME_OF_RAWDATA_FILE" --no-check-certificate # repeat this for every rawdata file -``` - -You will see a "Successfully created fsurdat files" message displayed at the end of the script if it ran through. - -The output will be a netCDF file similar to `surfdata_"your grid name"_hist_78pfts_CMIP6_simyr2000_c"yymmdd".nc`. - -**Congratulations!** You successfully created your surface data file! In the next step you will learn how to create your own atmospheric forcings. \ No newline at end of file diff --git a/docs/users_guide/case_creation/5_modifications_surface_domain_file.md b/docs/users_guide/case_creation/5_modifications_surface_domain_file.md deleted file mode 100644 index 17f78027a6..0000000000 --- a/docs/users_guide/case_creation/5_modifications_surface_domain_file.md +++ /dev/null @@ -1,39 +0,0 @@ -# Modification of the surface and domain file - - -## Handling negative longitudes and the landmask - -eCLM does not accept negative longitudes for the surface and domain file. In case you used a grid file to create your SCRIP grid file which used negative longitudes (instead of creating it through the `mkscripgrid.py` script), these need to be converted into the 0 to 360 degree system used by eCLM. You can use the `mod_domain.sh` script in the main directory `eCLM_static_file_workflow` to do this. - -Before executing the script adapt the paths to your [surface file](https://hpscterrsys.github.io/eCLM/users_guide/case_creation/4_create_surface_file.html#create-surface-file) and [domain file](https://hpscterrsys.github.io/eCLM/users_guide/case_creation/3_create_domain_file.html#create-domain-file). - -`mod_domain.sh` also replaces the `mask` and `frac` variables of your domain file with the information from a `landmask_file` (this `landmask.nc` file should contain the 2D variables `mask` and `frac` that contain your landmask (value 1 for land and 0 for ocean)). This step should not be necessary as you already swapped the `domain.lnd.*.nc` and `domain.ocn.*.nc` file when creating them. However, for some domains (e.g. the ICON grid) the mask from the rawdata may not correctly represent your landmask. Additionally, if you want to replace the surface parameters with higher resolution data (see below), you may need to update the landmask as well to match your surface parameters (e.g. coast lines may have changed). - -## Modifying surface parameters - -You may want to modify the default soil, landuse or other land surface data on the surface file if you have measurements or a different data source of higher resolution or similar available. -You can do this by accessing the relevant variables on the surface file. - -Variables you want to modify may include (non-exhaustive list): - -Soil: -- `PCT_SAND`: percentage sand at soil levels (10 levels are considered) -- `PCT_CLAY`: percentage clay at soil levels (10 levels are considered) -- `ORGANIC`: organic matter density at soil levels (10 levels are considered) - -Landuse at the landunit level ([Fig. 2](https://hpscterrsys.github.io/eCLM/users_guide/introduction_to_eCLM/introduction.html#fig2)): -- `PCT_NATVEG`: total percentage of natural vegetation landunit -- `PCT_CROP`: total percentage of crop landunit -- `PCT_URBAN`: total percentage of urban landunit -- `PCT_LAKE`: total percentage of lake landunit -- `PCT_GLACIER`: total percentage of glacier landunit -- `PCT_WETLAND`: total percentage of wetland landunit - -Types of crop and natural vegetation at the patch level ([Fig. 2](https://hpscterrsys.github.io/eCLM/users_guide/introduction_to_eCLM/introduction.html#fig2)): - -- `PCT_NAT_PFT`: percent plant functional type (PFT) on the natural veg landunit (% of landunit) (15 PFTs are considered see here for a list of PFTs) -- `PCT_CFT`: percent crop functional type (CFT) on the crop landunit (% of landunit) (2 CFTs are considered in SP mode, 64 CFTs are considered in BGC mode, see here for a list of CFTs) - -Land fraction: -- `LANDFRAC_PFT`: land fraction from PFT dataset -- `PFTDATA_MASK`: land mask from pft dataset, indicative of real/fake points \ No newline at end of file diff --git a/docs/users_guide/case_creation/6_create_atm_forcings.md b/docs/users_guide/case_creation/6_create_atm_forcings.md deleted file mode 100644 index 539fea3413..0000000000 --- a/docs/users_guide/case_creation/6_create_atm_forcings.md +++ /dev/null @@ -1,95 +0,0 @@ -# Create atmospheric forcing files - -There exist a few global standard forcing data sets that can be downloaded together with their domain file from the official data repository via this link. For beginners, it is easiest to start with these existing data files. - -- GSWP3 NCEP forcing dataset -- CRUNCEP dataset -- Qian dataset - - -To run with your own atmospheric forcing data, you need to set them up in NetCDF format that can be read by the atmospheric data model `DATM`. - -There is a list of eight variables that are expected to be on the input files. The names and units can be found in the table below (in the table TDEW and SHUM are optional fields that can be used in place of RH). The table also lists which of the fields are required and if not required what the code will do to replace them. If the names of the fields are different or the list is changed from the standard list of eight fields: FLDS, FSDS, PRECTmms, PSRF, RH, TBOT, WIND, and ZBOT, the resulting streams file will need to be modified to take this into account. - -```{list-table} Atmospheric forcing fields adapted from CESM1.2.0 User's Guide Documentation. -:header-rows: 1 -:name: tab1 - -* - Short-name - - Description - - Unit - - Required? - - If NOT required how replaced -* - FLDS - - incident longwave - - W/m2 - - No - - calculates based on Temperature, Pressure and Humidity (NOTE: The CRUNCEP data includes LW down, but by default we do NOT use it -- we use the calculated values) -* - FSDS - - incident solar - - W/m2 - - Yes - - / -* - FSDSdif - - incident solar diffuse - - W/m2 - - No - - based on FSDS -* - FSDSdir - - incident solar direct - - W/m2 - - No - - based on FSDS -* - PRECTmms - - precipitation - - mm/s - - Yes - - / -* - PSRF - - pressure at the lowest atm level - - Pa - - No - - assumes standard-pressure -* - RH - - relative humidity at the lowest atm level - - \% - - No - - can be replaced with SHUM or TDEW -* - SHUM - - specific humidity at the lowest atm level - - kg/kg - - Optional in place of RH - - can be replaced with RH or TDEW -* - TBOT - - temperature at the lowest atm level - - K (or can be C) - - Yes - - / -* - TDEW - - dew point temperature - - K (or can be C) - - Optional in place of RH - - can be replaced with RH or SHUM -* - WIND - - wind at the lowest atm level - - m/s - - Yes - - / -* - ZBOT - - observational height - - m - - No - - assumes 30 meters -``` - -All of the variables should be dimensioned: time, lat, lon, with time units in the form of "days since yyyy-mm-d hh:mm:ss" and a calendar attribute that can be "noleap" or "gregorian". There should be separate files for each month called `YYYY-MM.nc` where YYYY-MM corresponds to the four digit year and two digit month with a dash in-between. - -For single point cases where the atmospheric data has hourly or half-hourly temporal resolution, all data can be in the same monthly files (`YYYY-MM.nc`). For regional cases and if the data is at coarser temporal resolution, different time interpolation algorithms will be used for solar radiation, precipitation and the remaining input data so the data needs to be split into three files and placed into three different folders (`Precip/`, `Solar/`, `TPHWL/`). You also need a domain file to go with your atmospheric data which can be the same as the land domain file created in the previous workflow if the spatial resolution of your atmospheric data is the same as of your specified domain. - -For JSC users, an example python script to create forcings for a single-point case based on hourly observations can be found under `/p/scratch/cslts/shared_data/rlmod_eCLM`. - -Simply copy the script to your directory and adapt it to your own data. - -```sh -cp /p/scratch/cslts/shared_data/rlmod_eCLM/createnetCDF_forc_hourly_input.py $HOME -``` \ No newline at end of file diff --git a/docs/users_guide/case_creation/README.md b/docs/users_guide/case_creation/README.md deleted file mode 100644 index 30300e6881..0000000000 --- a/docs/users_guide/case_creation/README.md +++ /dev/null @@ -1,34 +0,0 @@ -# Creating a custom case - -This workflow will guide you through creating your own input datasets at a resolution of your choice for eCLM simulations. - -Throughout this process, you will use a range of different scripts to create the necessary files. - -```{figure} ../images/Build_custom_input.png -:height: 500px -:name: fig5 - -Overview of the work flow for the creation of custom surface datasets adapted from the CLM5.0 User's Guide. -``` -

- -This workflow is based on the following Github repository that contains all the necessary tools: https://github.com/HPSCTerrSys/eCLM_static_file_workflow. It follows the official CLM-workflow but makes a few adaptations. The basis is the clm5.0 release but there might be newer developments in the CTSM and CIME Github repositories. - -To get started, log into the JSC system and clone the repository for instance into your folder in `project1` that you created during the build of eCLM. - -```sh -cd /p/project1/projectID/user1 # replace projectID with your compute project and user1 with your username - -git clone https://github.com/HPSCTerrSys/eCLM_static_file_workflow.git -``` - -Sourcing the environment file that is contained in the repository will load all the required software modules. - -```sh -cd eCLM_static_file_workflow/ -source jsc.2023_Intel.sh -``` -You are now ready to start with the workflow. - -```{tableofcontents} -``` \ No newline at end of file diff --git a/docs/users_guide/case_examples/EURO-CORDEX.md b/docs/users_guide/case_examples/EURO-CORDEX.md deleted file mode 100644 index 1d774fccc0..0000000000 --- a/docs/users_guide/case_examples/EURO-CORDEX.md +++ /dev/null @@ -1,5 +0,0 @@ -# EURO-CORDEX - -```{warning} -Page under construction -``` \ No newline at end of file diff --git a/docs/users_guide/case_examples/NRW.md b/docs/users_guide/case_examples/NRW.md deleted file mode 100644 index c1853793af..0000000000 --- a/docs/users_guide/case_examples/NRW.md +++ /dev/null @@ -1,48 +0,0 @@ -# North-Rhine Westphalia - -This regional test case covers the state of North-Rhine Westphalia at a grid size of 300x300 grid cells. It runs in BGC mode (prognostic calculation of vegetation states at each time step for all 78 available plant functional types (PFTs) including crops). To set up eCLM and execute this test case, please follow the instructions below. - -```{attention} - The following only works for JSC users as the model inputs and forcing files are thus far not publicly available. -``` - -```{figure} ../images/nrw_boas.png -:height: 300px -:name: fig4 - -The DE-NRW domain extension and dominant land use type. Adapted from Boas et al (2023). -``` - -## 1. Copy the namelist files -For JSC users, all required namelist and input files to run this case are in the shared directory `/p/scratch/cslts/shared_data/rlmod_eCLM` - -```sh -cp -r /p/scratch/cslts/shared_data/rlmod_eCLM/example_cases/nrw_300x300 test_cases/ -cd test_cases/nrw_300x300 -``` - -## 2. Check case setup - -Print out the namelist files and look at the customized configurations. - -```sh -cat lnd_in -cat drv_in -``` - -For a complete list of `lnd_in`-namelist options, see -https://docs.cesm.ucar.edu/models/cesm2/settings/current/clm5_0_nml.html - -For a complete list of `drv_in`-namelist options, see -https://docs.cesm.ucar.edu/models/cesm2/settings/current/drv_nml.html - -## 3. Run the test case. - -Customize the copied job script `run-eclm-job.sh` as desired. In this example, it is already customized to this test case, you should just adapt the SBATCH parameters `--account` to your compute project and `--partition` to your system. As the NRW case is a regional domain, the number of processors is increased to 128 compared to the single-column case (SBATCH parameter `--ntasks-per-node=128`). - -```sh -sbatch run-eclm-job.sh -``` -To check the job status, run `sacct`. - -The model run is successful if the history files (`nrw_300x300.clm2.h0.*.nc`) have been generated. diff --git a/docs/users_guide/case_examples/README.md b/docs/users_guide/case_examples/README.md index 3c98c12a5a..ea43cd6a82 100644 --- a/docs/users_guide/case_examples/README.md +++ b/docs/users_guide/case_examples/README.md @@ -1,12 +1,4 @@ # Running example cases -Always load the eCLM environment before creating a case. This only needs to be done once per terminal session. - -```sh -source $HOME/load-eclm-variables.sh -``` - -## Cases - ```{tableofcontents} -``` \ No newline at end of file +``` diff --git a/docs/users_guide/case_examples/Wuestebach.md b/docs/users_guide/case_examples/Wuestebach.md index 8b26533b08..ae9272c3ba 100644 --- a/docs/users_guide/case_examples/Wuestebach.md +++ b/docs/users_guide/case_examples/Wuestebach.md @@ -1,5 +1,8 @@ # Single-Point Wuestebach +```{warning} TODO +``` + This test case at point scale covers the Wuestebach test site that is part of the TERENO network. Wuestebach is a forest site located in the Eifel region in Germany. To set up eCLM and run this test case, follow the instructions below. ```{figure} ../images/wtb_bogena.png @@ -9,34 +12,13 @@ This test case at point scale covers the Wuestebach test site that is part of th Location of the Wüstebach test site within the TERENO Rur/Lower Rhine Valley observatory. Adapted from Bogena et al (2010). ``` -## 1. Copy the namelist files -For JSC users, all required namelist and input files to run this case are in the shared directory `/p/scratch/cslts/shared_data/rlmod_eCLM` - -```sh -mkdir test_cases -cp -r /p/scratch/cslts/shared_data/rlmod_eCLM/example_cases/wtb_1x1 test_cases/ -cd test_cases/wtb_1x1 -``` - -## 1. Download and extract data files (**For non JSC users**) +## 1. Download Wuestebach data files -You can download all required files through the JSC datahub. ```sh -mkdir -p test_cases/1x1_wuestebach -wget https://datapub.fz-juelich.de/slts/eclm/1x1_wuestebach.tar.gz -tar xf 1x1_wuestebach.tar.gz -C test_cases/1x1_wuestebach -``` -The repository contains two directories. The `common` directory contains some general input files necessary to run eCLM cases. The `wtb_1x1` directory contains the case specific domain and surface files as well as atmospheric forcing data and a script for namelist generation. - -```sh -# Generate namelists -cd test_cases/1x1_wuestebach -export ECLM_SHARED_DATA=$(pwd) -cd wtb_1x1 -clm5nl-gen wtb_1x1.toml - -# Validate namelists -clm5nl-check . +git clone https://icg4geo.icg.kfa-juelich.de/ExternalReposPublic/tsmp2-static-files/extpar_eclm_wuestebach_sp.git +cd extpar_eclm_wuestebach_sp/static.resources +generate_wtb_namelists.sh 1x1_wuestebach +cd 1x1_wuestebach ``` ## 2. Check the case setup @@ -55,58 +37,8 @@ cat drv_in ## 3. Run the test case -Customize the copied job script `run-eclm-job.sh` as desired. In this example, it is already customized to this test case, you should just adapt the SBATCH parameters `--account` to your compute project and `--partition` to your system. As Wuestebach is a single-column case, the number of processors should be set to 1 (SBATCH parameter `--ntasks-per-node=1`). - -**For non JSC users**: Create a job script in your case directory with: - -```sh -cat << EOF > run-eclm-job.sh -``` -Adapt the SBATCH parameters and then copy the following in the shell file: - -```sh -#!/usr/bin/env bash -#SBATCH --job-name=wtb_1x1 -#SBATCH --nodes=1 -#SBATCH --ntasks-per-node=1 -#SBATCH --account=jibg36 -#SBATCH --partition=batch -#SBATCH --time=1:00:00 -#SBATCH --output=logs/%j.eclm.out -#SBATCH --error=logs/%j.eclm.err - -ECLM_EXE=${eCLM_ROOT}/install/bin/eclm.exe -if [[ ! -f $ECLM_EXE || -z "$ECLM_EXE" ]]; then - echo "ERROR: eCLM executable '$ECLM_EXE' does not exist." - exit 1 -fi - -# Set PIO log files -if [[ -z $SLURM_JOB_ID || "$SLURM_JOB_ID" == " " ]]; then - LOGID=$(date +%Y-%m-%d_%H.%M.%S) -else - LOGID=$SLURM_JOB_ID -fi -mkdir -p logs timing/checkpoints -LOGDIR=$(realpath logs) -comps=(atm cpl esp glc ice lnd ocn rof wav) -for comp in ${comps[*]}; do - LOGFILE="$LOGID.comp_${comp}.log" - sed -i "s#diro.*#diro = \"$LOGDIR\"#" ${comp}_modelio.nml - sed -i "s#logfile.*#logfile = \"$LOGFILE\"#" ${comp}_modelio.nml -done - -# Run model -srun $ECLM_EXE -EOF -``` - -Then you can submit your job: - -```sh -sbatch run-eclm-job.sh +```bash +mpirun -np 1 eclm.exe ``` -To check the job status, run `sacct`. - The model run is successful if the history files (`wtb_1x1.clm2.h0.*.nc`) have been generated. diff --git a/docs/users_guide/images/Putty_X11.png b/docs/users_guide/images/Putty_X11.png deleted file mode 100644 index 138a72b495..0000000000 Binary files a/docs/users_guide/images/Putty_X11.png and /dev/null differ diff --git a/docs/users_guide/images/load_env.png b/docs/users_guide/images/load_env.png deleted file mode 100644 index fea39337de..0000000000 Binary files a/docs/users_guide/images/load_env.png and /dev/null differ diff --git a/docs/users_guide/images/winSCP.png b/docs/users_guide/images/winSCP.png deleted file mode 100644 index 36d9e5f590..0000000000 Binary files a/docs/users_guide/images/winSCP.png and /dev/null differ diff --git a/docs/users_guide/installation/README.md b/docs/users_guide/installation/README.md deleted file mode 100644 index 5599edd686..0000000000 --- a/docs/users_guide/installation/README.md +++ /dev/null @@ -1,13 +0,0 @@ -# Installing eCLM - -The easiest way to install eCLM is through [TSMP2 build system](https://github.com/HPSCTerrSys/TSMP2). - -```sh -# Download TSMP2 -git clone https://github.com/HPSCTerrSys/TSMP2.git -cd TSMP2 - -# Build and install eCLM -./build_tsmp2.sh --eCLM -``` - diff --git a/docs/users_guide/introduction/introduction.md b/docs/users_guide/introduction/introduction.md deleted file mode 100644 index c88387764b..0000000000 --- a/docs/users_guide/introduction/introduction.md +++ /dev/null @@ -1,44 +0,0 @@ -# The Community Land Model - -The Community Land Model is the land model for the Community Earth System Model (CESM). It focuses on modeling the land as the critical interface through which humanity affects, adapts to, and mitigates global environmental change. It includes a comprehensive representation of land biogeophysics, hydrology, plant physiology, biogeochemistry, anthropogenic land use, and ecosystem dynamics across a variety of spatial and temporal scales. The central theme is that terrestrial ecosystems, through their cycling of energy, water, chemical elements, and trace gases, are important determinants of climate. -The model represents several aspects of the land surface including surface heterogeneity and consists of components or submodels related to land biogeophysics, the hydrologic cycle, biogeochemistry, human dimensions, and ecosystem dynamics. Specific processes that are represented include: - -- Vegetation composition, structure, and phenology -- Absorption, reflection, and transmittance of solar radiation -- Absorption and emission of longwave radiation -- Momentum, sensible heat (ground and canopy), and latent heat (ground evaporation, canopy evaporation, transpiration) fluxes -- Heat transfer in soil and snow including phase change -- Canopy hydrology (interception, throughfall, and drip) -- Snow hydrology (snow accumulation and melt, compaction, water transfer between snow layers) -- Soil hydrology (surface runoff, infiltration, redistribution of water within the column, sub-surface drainage, groundwater) -- Plant hydrodynamics -- Stomatal physiology and photosynthesis -- Lake temperatures and fluxes -- Dust deposition and fluxes -- Routing of runoff from rivers to ocean -- Volatile organic compounds emissions -- Urban energy balance and climate -- Carbon-nitrogen cycling -- Dynamic landcover change -- Land management including crops and crop management and wood harvest -- Ecosystem Demography (FATES, optional) - -```{figure} ../images/CLM5_processes_Lawrence2019.png -:height: 400px -:name: fig1 - -Overview of land biogeophysical, biogeochemical and landscape processes simulated by CLM5 Lawrence et al. (2019). -``` -

- -Each time step the model solves the surface energy balance, water balance, and carbon exchange. Submodels of CLM include biophysics and biogeochemistry. It represents land surface heterogeneity through a subgrid tiling structure and different plant functional types that have unique parameters in terms of optical properties, morphology, photosynthesis. It can be run with prescribed vegetation states using satelllite phenology (SP mode) or prognostic vegetation states and biogeochemistry (BGC mode). - -```{figure} ../images/CLM5_subgrid_structure_v1.png -:height: 400px -:name: fig2 - -Configuration of the CLM subgrid hierarchy. Box in upper right shows hypothetical subgrid distribution for a single grid cell. Note that the Crop land unit is only used when the model is run with the crop model active. Abbreviations: TBD – Tall Building District; HD – High Density; MD – Medium Density, G – Glacier, L – Lake, U – Urban, C – Crop, V – Vegetated, PFT – Plant Functional Type, Irr – Irrigated, UIrr – Unirrigated. Red arrows indicate allowed land unit transitions. Purple arrows indicate allowed patch-level transitions. From CLM5 Technical Note. -``` -

- -More background information on CLM (and it’s latest release CLM5) including documentation and publications can be found here. \ No newline at end of file diff --git a/docs/users_guide/introduction/quick_start.md b/docs/users_guide/introduction/quick_start.md new file mode 100644 index 0000000000..11092e46f6 --- /dev/null +++ b/docs/users_guide/introduction/quick_start.md @@ -0,0 +1,85 @@ +# Quick Start + +This guide lets you install eCLM on the target machine. + +## Setting up eCLM on your local machine + +Download [Podman]. + +1. Fetch the eCLM container image. On your terminal, run the ff.: + +```sh +mkdir $HOME/eclm_share +podman run --name eclm -it --mount type=bind,src=$HOME/eclm_share,target=/share hpscterrsys/eclm:latest +``` + +Change permissions. + +```sh +sudo chown -R user /share +sudo chgrp -R hpscterrsys /share +``` + +2. Build eCLM. + +```sh +cd /share +build_eclm.sh +``` + +3. Set up a simulation experiment. + +```sh +cd /share +git clone https://icg4geo.icg.kfa-juelich.de/ExternalReposPublic/tsmp2-static-files/extpar_eclm_wuestebach_sp.git +cd extpar_eclm_wuestebach_sp/static.resources +generate_wtb_namelists.sh 1x1_wuestebach +``` + +4. Run eCLM. + +```sh +cd 1x1_wuestebach +mpirun -np 1 eclm.exe +``` + +## Setting up eCLM on HPC systems + +The steps are similar to above. The only difference is the build step and running step. + +1. Download TSMP2 build system. + +```sh +# eCLM can be easily built via the TSMP2 build system. The following step will download TSMP2. +git clone https://github.com/HPSCTerrSys/TSMP2.git +``` + +2. Build eCLM + +```sh +# Build eCLM +cd TSMP2 +./build_tsmp2.sh eCLM +``` + +3. Set up a simulation experiment. + +```sh +cd /share +git clone https://icg4geo.icg.kfa-juelich.de/ExternalReposPublic/tsmp2-static-files/extpar_eclm_wuestebach_sp.git +cd extpar_eclm_wuestebach_sp/static.resources +generate_wtb_namelists.sh 1x1_wuestebach +``` + +4. Run eCLM. + +```sh +cd 1x1_wuestebach +mpirun -np 1 eclm.exe +``` + +[Podman]: https://docs.podman.io/en/latest/Tutorials.html +[Docker]: https://docs.docker.com/get-started +[VirtualBox]: https://www.virtualbox.org +[UTM]: https://mac.getutm.app +[Ubuntu 24.04 LTS]: https://hub.docker.com/_/ubuntu diff --git a/docs/users_guide/misc_tutorials/README.md b/docs/users_guide/misc_tutorials/README.md new file mode 100644 index 0000000000..95ccf907b3 --- /dev/null +++ b/docs/users_guide/misc_tutorials/README.md @@ -0,0 +1,7 @@ +# Miscellaneous tutorials + +Special use cases of eCLM go into this section. + + +```{tableofcontents} +``` diff --git a/docs/users_guide/introduction/perturbation.md b/docs/users_guide/misc_tutorials/perturbation.md similarity index 100% rename from docs/users_guide/introduction/perturbation.md rename to docs/users_guide/misc_tutorials/perturbation.md diff --git a/docs/users_guide/introduction/soil_hydraulic_parameters.md b/docs/users_guide/misc_tutorials/soil_hydraulic_parameters.md similarity index 100% rename from docs/users_guide/introduction/soil_hydraulic_parameters.md rename to docs/users_guide/misc_tutorials/soil_hydraulic_parameters.md diff --git a/docs/users_guide/running_cases/case_customization.md b/docs/users_guide/running_cases/case_customization.md deleted file mode 100644 index 2284f5ea46..0000000000 --- a/docs/users_guide/running_cases/case_customization.md +++ /dev/null @@ -1,114 +0,0 @@ -# Case customization - -eCLM uses various **namelist files** that handle different settings and configurations for running a case. -Namelist files are text files, where the user can provide values for various input fields, called **namelist items**. -Namelist items are parsed by eCLM and connected internal variables are set according to their values. -eCLM's namelist files are very similar to the namelist files used in CLM5 (only the editing is different) so that you can refer to Section 1.2.3 and 1.2.4 of the CLM5 User's Guide for more detailed information. - -The eCLM namelist files are: -- Land model namelist `lnd_in` (see here for definitions of namelist items) -- Data atmosphere namelist `datm_in` (see here for definitions of namelist items) -- River runoff model namelist `mosart_in` (see here for definitions of namelist items) -- Driver namelist `drv_in` (see here for definitions of namelist items) -- Driver fields namelist `drv_flds_in` (see here for definitions of namelist items) - -Many of the namelist options are set by default and you do not need to change them, others you want to adapt when running your own case. - -You can customize your case in various ways: - -1. Customizing input data -2. Customizing case run -3. Customizing case output - -## 1. Customizing input data - -The `lnd_in` file is the primary configuration file for the land model component in eCLM. It includes settings related to various processes and configurations specific to land surface modeling including which files are used as input data to the land and atmosphere. - -Important namelist parameters to adjust for model input in `lnd_in` are: - -- `fatmlndfrc`: Path to your domain file (holding grid information). -- `fsurdat`: Path to the surface data file (vegetation, soil types, etc.). -- `finidat`: Path to the initial conditions file (empty if doing a cold start). -- `datm_mode`: The mode for atmospheric forcing. -- `paramfile`: Path to your parameter file. -- `streams`: Paths to files containing atmospheric forcing data. - -Also, make sure the paths pointing to the various common inputdata is still correct. - -Additionally, the `datm_in` holds information on the atmospheric data used to force your simulation and should be adapted to your specific case. - -Important namelist parameters in `datm_in` are: - -- `domainfile`: Path to your domain file (holding grid information tht will serve as target grid for the atmospheric data) -- `mapalgo`: Sets the spatial interpolation method (usually the default is fine). -- `tintalgo`: Sets the time interpolation algorithm (usually the default is fine). - -You should also adapt the stream files when using your own atmospheric forcings. - -- `streams`: Data stream files for the different atmospheric input data - -## 2. Customizing case run - -Customizing your case run involves defining the period over which you want to run, the starting date etc. - -Important namelist parameter in the `drv_in` are: - -- `start_ymd`: The start date of the simulation in the format YYYYMMDD. -- `stop_n`: The number of time units (days, months, years) for the simulation. -- `stop_option`: The time unit for stop_n (e.g., 'nyears', 'nmonths', 'ndays'). -- `stop_ymd`: The end date of the simulation. -- `restart_file`: The name of your restart file (file from which the simulation should start/continue when not doing a cold start) -- `restart_n`: The number of time units (days, months, years) for the restart file. -- `restart_option`: The time unit for restart_n (e.g., 'nyears', 'nmonths', 'ndays'). -- `restart_ymd`: The date for writing a restart file. - -## 3. Customizing case output - -To customize your simulation output, you can specify the history field -options in the namelist file `lnd_in`. -By default, there is one stream of monthly data files called history -files. -History files are in NetCDF format. -The field options to customize are: - -- `hist_fincl1`: The list of history variables that you want to analyze -- `hist_mfilt`: The number of records within one output file. Default is 1. -- `hist_nhtfrq`: The frequency at which data is recorded and written to a file. Default: 0 – monthly output (monthly averages), positive values: number of model timesteps (half-hourly), negative values: absolute value in hours for output record - -Here are two examples of a customized output: - -**Ex. 1: Daily output with a year’s worth of daily records in a file** -``` -hist_fincl1 = "TLAI","TSOI","TOTSOMC" -hist_mfilt = 365 -hist_nhtfrq = -24 -``` -**Ex 2.: Monthly output with each month written to a separate file** -``` -hist_fincl1 = "TLAI","GPP","TOTVEGC" -hist_mfilt = 1 -hist_nhtfrq = 0 -``` - -To add to the list of output variables, simply extend `hist_fincl1` with the desired history fields. A list of all available CLM5 history fields can be found here. - - ```{tip} -Additional streams can be added using `hist_fincl2` through `hist_fincl6`, each can have different output frequencies and averaging methods. By default, only the primary history files are active. The primary history files are monthly by default, and other streams are daily but you can adapt this. -In addition, it can be useful to remove all (or the default) history fields. The option `hist_empty_htapes` allows you to turn off all default output. You can then still output your own reduced list of history fields using `hist_fincl1`. - -The default averaging depends on the specific field but for most of the fields it is the average over the output interval. Other averaging can be specified in two ways. -- By adding an averaging flag to the end of the field name after a colon (for example `TSOI:X`, would output the maximum of soil temperature) -- Using the `hist_avgflag_pertape` argument - -For more details on the usage see here. -``` - -After having adjusted your namelist files you may also want to adjust the job script `run-eclm-job.sh` e.g. to change the job name or to allocate more computing resources when running a larger case. - -You can then submit the job. - -```sh -sbatch run-eclm-job.sh -``` - -You can monitor your job using `sacct` or `squeue -u $USER`.