Push notifications in your browser are not yet configured.
You are not logged in, you may not see all content and functionalities. If you have an account, please login .
CCLM in levante at DKRZ
Hi Community,
has anybody started running CCLM on the new machine at DKRZ , the levante?
Could you share your Fopts files?
Cheers,
Bijan
For those who are interested, I used the following setting to install int2lm and cclm on Levante.
“Note that the paths can change because DKRZ is still recreating the entire software tree.
It should become more stable in the near future”. The cdo and nco are not installed yet but will be soon there.
NETCDFC _ROOT = /sw/spack-levante/netcdf-c-4.8.1-qk24yp
NETCDFF_ROOT = /sw/spack-levante/netcdf-fortran-4.5.3-k6xq5g
HDF5_ROOT = /sw/spack-levante/hdf5-1.12.1-akf2kp
SZIP_ROOT = /sw/spack-levante/libaec-1.0.5-r5sdw5
and change the following line in Fopts:
NC_LIB += -L$( SZIP _ROOT)/lib -lsz -lm
to
NC_LIB += -L$( SZIP _ROOT)/lib64 -lsz -lm
Cheers,
Bijan
Thanks! This looks promising.
What modules do you load?
Can you attach the whole Fopts, please?
I tried the libraries, they work fine.
But which modules did you load?
I compiled int2lm using mpif90, but I can’t run it since modules like “mxm” are not yet installed.
Thank you for helping.
Mostafa
I have found the modules. It is fine now.
Could you share them?
Thank you
openmpi/4.1.2-intel-2021.5.0
intel-oneapi-compilers/2022.0.1-gcc-11.2.0
Thank you. It compiled fine. However I still have problems with MPI Runtime Settings.
Could you share yours?
I am getting an error such as:
Caught signal 11 (Segmentation fault: address not mapped to object at address
Thank you
Hi Community
Thanks for the useful information, I successfully compiled the CCLM and int2lm. But while I was trying to run the model, it gave me this error:
sbatch: unrecognized option ‘—purge-slurm-env‘
Try “sbatch —help” for more information
Does anyone know about this? it is because of the subchain script? Do we need to update the subchain script?
Thank you very much in advance.
Cheers
Mingyue
Hi Mingyue,
I commented that in int2lm code (i.e. removed that line) and got no errors. It is a slurm flag I think.
Cheers,
Bijan
Hi Bijan
Thank you very much, I figured it out later I posted my confusion, but while running the model, it still gives errors such as “lib” missing, I think this is related to the Levante system build-up.
Cheers
Mingyue
Hi, Community
Thanks for the useful information, I successfully compiled the CCLM and int2lm. Int2lm runs without any issues, but cclm stops with an error: —————————————————————————————— 0: * PROGRAM TERMINATED BECAUSE OF ERRORS DETECTED 0: * IN ROUTINE : output_data 0: * 0: * ERROR CODE is 2035 0: * Error writing netcdf 2D variable 0: ——————————————————————————————
I used COSMO - CLM v5.0, I want to ask is there anyone who experienced the same issue? or perhaps you have some idea on how to solve this problem?
Thank you all very much in advance.
Mingyue
Two possibilites, dear Mingyue,
A) There is a NETCDF -library problem. Hopefully, you have used the compiler NETCDF settings given by ‘nc-config —all’
B) There is a configuration problem. Maybe, you start with an output configuration where only one variable is written out in only one output stream.
I am convinced that the error code is thrown out from the NETCDF -Libraries and not the COSMO model itself.
Bye,
Ronny
HI Ronny,
Thanks for the quick reply.
Unfortunately I do not remember that I have used or changed some line that contains the “nc-config —all” in any files.
Perhaps you know where I should use this command? in the Fobts file or the general my home directory?
Cheers
Mingyue
Hey Mingyue,
first go for the option B). Throw away all output except for one variable and one output block.
On which machine do you work?
Bye,
Ronny
Hi Ronny
I am on Levante.
And what do you mean by “ Throw away all output except for one variable and one output block “? Sorry that I didn’t understand.
Cheers
Mingyue
Hello Mingyue,
I mean the namelist of COSMO - CLM
!!out01: &GRIBOUT yform_write='ncdf', ncomb=${NSTART},${NNEXT},${NOUT_INC[$((NOUTDIR+=1))]}, yvarml='FRESHSNW','QV_S','T_S','T_SNOW','T_SO','W_I','W_SNOW','W_SO', 'PP','QC','QI','QR','QS','Z0','QV','T','U','V','W', !QG is only available in high-resolution runs yvarpl=' ', yvarzl=' ', luvmasspoint=.FALSE., lcheck = .FALSE., lwrite_const= ${LWRITE_CONST}, ydir='${OUTDIR}/${YYYY_MM}/out$(printf %02d $NOUTDIR)/', ytunit='d', /END
Throw away all variables in yvarml and leave only one.
Bye,
Ronny
user#270 wrote:
> Hello Mingyue,
>
> I mean the namelist of COSMO-CLM
>
>
!!out01: > &GRIBOUT > yform_write='ncdf', > ncomb=${NSTART},${NNEXT},${NOUT_INC[$((NOUTDIR+=1))]}, > yvarml='FRESHSNW','QV_S','T_S','T_SNOW','T_SO','W_I','W_SNOW','W_SO', > 'PP','QC','QI','QR','QS','Z0','QV','T','U','V','W', !QG is only available in high-resolution runs > yvarpl=' ', > yvarzl=' ', > luvmasspoint=.FALSE., > lcheck = .FALSE., > lwrite_const= ${LWRITE_CONST}, > ydir='${OUTDIR}/${YYYY_MM}/out$(printf %02d $NOUTDIR)/', > ytunit='d', > /END >
>
> Throw away all variables in yvarml and leave only one.
>
> Bye,
> Ronny
Hi, Ronny,
I commented out all the output but ‘ FRESHSNW ’ in output01, (all the variables from the other output channel are also commented out), like this &GRIBOUT ! out01 yform_write=‘ncdf’, ncomb=${NSTART},${NNEXT},${NOUT_INC[$(( NOUTDIR +=1))]},
! uncomment the next but one line if you plan double nesting yvarml=‘ FRESHSNW ’, !‘QV_S’,‘T_S’,‘T_SNOW’,‘T_SO’,‘W_I’,‘W_SNOW’,‘W_SO’, !‘PP’,‘QC’,‘QI’,‘QR’,‘QS’,‘QV’,‘T’,‘U’,‘V’, yvarpl=’ ‘, yvarzl=’ ‘, luvmasspoint=.FALSE., lcheck = .FALSE., lwrite_const= ${LWRITE_CONST}, ydir=’${OUTDIR}/${YYYY_MM}/out$(printf %02d $NOUTDIR)/’, ytunit=‘d’, /END
! &GRIBOUT ! out02
! yform_write=‘ncdf’,
! ncomb=${NSTART},${NNEXT},${NOUT_INC[$(( NOUTDIR +=1))]},
! yvarml=‘W’,‘P’,
! yvarpl=’ ‘,
! yvarzl=’ ‘,
! luvmasspoint=.TRUE.,
! lcheck = .FALSE.,
! lwrite_const= ${LWRITE_CONST},
! ydir=’${OUTDIR}/${YYYY_MM}/out$(printf %02d $NOUTDIR)/’,
! ytunit=‘d’,
! /END
! &GRIBOUT ! out03
! yform_write=‘ncdf’,
! ncomb=${NSTART},${NNEXT},${NOUT_INC[$(( NOUTDIR +=1))]},
! yvarml=‘ RAIN _CON’,‘ SNOW _CON’,‘ RAIN _GSP’,‘ SNOW _GSP’,‘ TOT _PREC’,
! yvarpl=’ ‘,
! yvarzl=’ ‘,
! luvmasspoint=.TRUE.,
! lcheck = .FALSE.,
! lwrite_const= ${LWRITE_CONST}, ! ydir=’${OUTDIR}/${YYYY_MM}/out$(printf %02d $NOUTDIR)/’,
! ytunit=‘d’, ! /END
and then I got the following error:
137: ——————————————————————————————
137: * PROGRAM TERMINATED BECAUSE OF ERRORS DETECTED
137: * IN ROUTINE : organize_data: input
138: ——————————————————————————————
Cheers
Mingyue
You are using the subchain. Thus, please adapt the job_settings and check for
NESTING_STREAM=( 1 ) # If you want to use an output stream to save quantities as input for further downscaling put the number of the output streams here. HOUT_INC=(03:00:00 01:00:00 06:00:00 01:00:00 01:00:00 03:00:00 06:00:00 24:00:00 24:00:00 01:00:00 01:00:00 06:00:00 06:00:00) # time increment for each GRIBOUT hh:mm:ss
Modify it to
HOUT_INC=(03:00:00)
. Please also sent the file INPUT _IO generated in the directory /joboutputs/cclm/19XX_01/.Bye Ronny
Dear Ronny
I did as you mentioned, I am using COSMO - CLM v5.0, thus there isn’t a variable called NESTING _STREAM in the job_settings. Nevertheless, I tried to change HOUT _INC=(7 03:00:00 24:00:00 01:00:00 03:00:00 06:00:00 06:00:00 24:00:00) into HOUT _INC=(1 03:00:00) I think here the 7 means NESTING _STREAM=( 1 ).
It stopped because of the same reason: Error writing netcdf 2D variable, can we now make sure that it is because of the netcdf libraries you mentioned before, could you please let me how should I proceeding in this point, where should I involve this “nc-config —all”.
Thank you very much in advance!
Regards
Mingyue
If you compile the COSMOv5 on MISTRAL , you have used a specific Fopts-File. If you are familiar with this files, then i would highly recommend to copy this files from COSMO6 Starter package and compile COSMOv5 using them. If you are not familiar with the details of this files, I would highly recommend to download COSMO6 starter package and to prevent from further investigation with COSMOv5.
Dear Community,
I am working together with Mingyue to solve the problem she mentioned before. We successfully compiled the COSMOv5 and also made it run on Levante. Everything seems to work except for the writing of NetCDF restart files. The creation of NetCDF-output and every other step works without any problems. The CCLM -job runs until almost the end and crashes when writing the restart file. In a test, it worked when writing the restart file as binary but this is not really an option for us. Also using COSMOv6 is only a last solution because we want to continue our 2500 years of simulation (almost 1000 years already done) with as few changes as possible. 0: OPEN : ncdf-file: 0: /work/bb1201/b381165_bb1152//mythos500bc_CCLM/restarts/lrfd7476030100o.nc 0: —————————————————————————————— 0: * PROGRAM TERMINATED BECAUSE OF ERRORS DETECTED 0: * IN ROUTINE : output_data 0: * 0: * ERROR CODE is 2035 0: * Error writing netcdf 2D variable 0: —————————————————————————————— 0: ————————————————————————————————————— 0: MPI _ABORT was invoked on rank 0 in communicator MPI _COMM_WORLD 0: with errorcode 2035. 0: 0: NOTE : invoking MPI _ABORT causes Open MPI to kill all MPI processes. 0: You may or may not see output from other processes, depending on 0: exactly when Open MPI kills them. 0: —————————————————————————————————————
srun: Job step aborted: Waiting up to 302 seconds for job step to finish. 0: slurmstepd: error: *** STEP 1344625.0 ON l30116 CANCELLED AT 2022-08-03T09:49:15 ***
srun: error: l30116: task 0: Exited with exit code 243
srun: launch/slurm: _step_signal: Terminating StepId=1344625.0
srun: error: l30116: tasks 1-143: Killed
srun: error: l30117: tasks 144-287: Killed
Does anyone know about the differences between writing the output and writing the restart file? Or did anyone have to deal with similar problems?
We would be very happy about any suggestion and help.
Thank you very much and kind regards,
Eva
HEllo,
you did everything right with your Fopts file. Nice to hear, that now the output of netcdf-files works properly. However, restart-netcdf is not written. Regarding this I would recommend two things:
1) Keep sure that you use the most recent version of COSMOv5 provided by Burhkardt Rockel on our REDC homepages (as the netcdf-restart functionality is very new and made some problems).
2) Be happy with what you achieved so far and change to binary restarts. It will not affect your integration.
Bye,
Ronny