Cutting Input files – in #10: INT2LM

in #10: INT2LM

<p> Dear all, </p> <p> Due to storage limits, I wanted to check whether: a. it is possible to download the files ( <span class="caps"> ERA </span> <span class="caps"> INTERIM </span> or <span class="caps"> ECHAM </span> -6) already cut for any required domain\area. B. whether int2lm can read such files (or does it need “complete” files). C.or should we cut them afterwards and then with which tool would you recommend us to use? </p> <p> We simply want to use the input files afterwards for analysis, so we prefer to keep them here </p> <p> What would you recommend us to do? </p> <p> Thank you, <br/> Sincerely yours, </p> <p> Giora </p>

  @redc_migration in #1828723

<p> Dear all, </p> <p> Due to storage limits, I wanted to check whether: a. it is possible to download the files ( <span class="caps"> ERA </span> <span class="caps"> INTERIM </span> or <span class="caps"> ECHAM </span> -6) already cut for any required domain\area. B. whether int2lm can read such files (or does it need “complete” files). C.or should we cut them afterwards and then with which tool would you recommend us to use? </p> <p> We simply want to use the input files afterwards for analysis, so we prefer to keep them here </p> <p> What would you recommend us to do? </p> <p> Thank you, <br/> Sincerely yours, </p> <p> Giora </p>

Cutting Input files

re wrote

Dear all,

Due to storage limits, I wanted to check whether: a. it is possible to download the files ( ERA INTERIM or ECHAM -6) already cut for any required domain\area. B. whether int2lm can read such files (or does it need “complete” files). C.or should we cut them afterwards and then with which tool would you recommend us to use?

We simply want to use the input files afterwards for analysis, so we prefer to keep them here

What would you recommend us to do?

Thank you,
Sincerely yours,

Giora

View in channel
<p> Dear Giora, </p> <p> no, to my knowledge there are no “cutted” <span class="caps"> ERA </span> , <span class="caps"> ERA </span> -Int or <span class="caps"> ECHAM </span> -6 files available, only files with global data. </p> <p> <span class="caps"> ERA </span> and <span class="caps"> ERA </span> -Int are available from <br/> /pool/data/CCLM/reanalyses <br/> on <span class="caps"> BLIZZARD </span> , which you can copy if you are a member of group “ecrana”. <br/> Burkhardt might tell you more details. </p> <p> <span class="caps"> ECHAM </span> -6 (=MPI- <span class="caps"> ESM </span> -LR) data is a little bit more complicate. <br/> They are open for all <span class="caps"> CLM </span> -Community members but they have been archived in the <span class="caps"> HPSS </span> system at <span class="caps"> DKRZ </span> . <br/> Thus, they have to be restored first and then they can be copied. <br/> For the historical period (1949 – 2005), 2 realizations (r1 and r2) of <span class="caps"> MPI </span> - <span class="caps"> ESM </span> -LR data are available. <br/> RCP scenarios, period 2006-2100 <br/> RCP2.6: 2 realizations, r1 and r2 <br/> RCP4.5: 2 realizations, r1 and r2 <br/> RCP8.5: 2 realizations, r1 and r2 </p> <p> Anyway, you need an account at <span class="caps"> DKRZ </span> . </p> <p> Yes, INT2LM is able to treat such regionally cutted files. Such cutted files should get “cas” as the “header” of the file name, instead of “caf” for the global files. <br/> Example: cas1990010100.nc for the 1st of Jan, 1990, 00:00 o’clock </p> <p> I would use “cdo sellonlatbox” or a corresponding “nco” command ( I am not so familiar with <span class="caps"> NCO </span> s, perhaps Burkhardt can help again) for cutting. </p> <p> Hans-Juergen </p>

  @hans-jürgenpanitz in #0077281

<p> Dear Giora, </p> <p> no, to my knowledge there are no “cutted” <span class="caps"> ERA </span> , <span class="caps"> ERA </span> -Int or <span class="caps"> ECHAM </span> -6 files available, only files with global data. </p> <p> <span class="caps"> ERA </span> and <span class="caps"> ERA </span> -Int are available from <br/> /pool/data/CCLM/reanalyses <br/> on <span class="caps"> BLIZZARD </span> , which you can copy if you are a member of group “ecrana”. <br/> Burkhardt might tell you more details. </p> <p> <span class="caps"> ECHAM </span> -6 (=MPI- <span class="caps"> ESM </span> -LR) data is a little bit more complicate. <br/> They are open for all <span class="caps"> CLM </span> -Community members but they have been archived in the <span class="caps"> HPSS </span> system at <span class="caps"> DKRZ </span> . <br/> Thus, they have to be restored first and then they can be copied. <br/> For the historical period (1949 – 2005), 2 realizations (r1 and r2) of <span class="caps"> MPI </span> - <span class="caps"> ESM </span> -LR data are available. <br/> RCP scenarios, period 2006-2100 <br/> RCP2.6: 2 realizations, r1 and r2 <br/> RCP4.5: 2 realizations, r1 and r2 <br/> RCP8.5: 2 realizations, r1 and r2 </p> <p> Anyway, you need an account at <span class="caps"> DKRZ </span> . </p> <p> Yes, INT2LM is able to treat such regionally cutted files. Such cutted files should get “cas” as the “header” of the file name, instead of “caf” for the global files. <br/> Example: cas1990010100.nc for the 1st of Jan, 1990, 00:00 o’clock </p> <p> I would use “cdo sellonlatbox” or a corresponding “nco” command ( I am not so familiar with <span class="caps"> NCO </span> s, perhaps Burkhardt can help again) for cutting. </p> <p> Hans-Juergen </p>
HP replied

Dear Giora,

no, to my knowledge there are no “cutted” ERA , ERA -Int or ECHAM -6 files available, only files with global data.

ERA and ERA -Int are available from
/pool/data/CCLM/reanalyses
on BLIZZARD , which you can copy if you are a member of group “ecrana”.
Burkhardt might tell you more details.

ECHAM -6 (=MPI- ESM -LR) data is a little bit more complicate.
They are open for all CLM -Community members but they have been archived in the HPSS system at DKRZ .
Thus, they have to be restored first and then they can be copied.
For the historical period (1949 – 2005), 2 realizations (r1 and r2) of MPI - ESM -LR data are available.
RCP scenarios, period 2006-2100
RCP2.6: 2 realizations, r1 and r2
RCP4.5: 2 realizations, r1 and r2
RCP8.5: 2 realizations, r1 and r2

Anyway, you need an account at DKRZ .

Yes, INT2LM is able to treat such regionally cutted files. Such cutted files should get “cas” as the “header” of the file name, instead of “caf” for the global files.
Example: cas1990010100.nc for the 1st of Jan, 1990, 00:00 o’clock

I would use “cdo sellonlatbox” or a corresponding “nco” command ( I am not so familiar with NCO s, perhaps Burkhardt can help again) for cutting.

Hans-Juergen

View in channel
<p> Yes, INT2LM can read global data cut out for a sub region. In older INT2LM version ( I am not sure until which version) it was an advantage to cut out the region before running the INT2LM since this would save time because the old versions always took the whole data into account for interpolation. In the versions we currently use the global data are cut in the INT2LM program first before the interpolation is done. <br/> If you have an account at <span class="caps"> DKRZ </span> you can do the cutting there, otherwise you need to transfer the data to your computer first. You may do the latter in a script month by month in order to keep the amount of disk space low at your institute. There is no automatic cutting procedure at <span class="caps"> DKRZ </span> presently. <br/> In oder to save space you can proceed as follows (assume for example your the original file is caf1992010100.nc): </p> <p> Use cdo or ncks to cut out the region you need. Actually cut out a bit larger region since the interpolation needs additional grid points at the boundaries. <br/> For a region with lower left (longitude1, latitude1) and upper right (longitude2, latitude2) perform either <br/> <pre> cdo sellonlatbox,longitude1,longitude2,latitude1,latitude2 caf1992010100.nc cas1992010100.nc ncks -h -d lon,longitude1,longitude2 -d lat,latitude1,latitude2 caf1992010100.nc cas1992010100.nc</pre> In case you use ncks: The coordinates should be <span class="caps"> REAL </span> values not <span class="caps"> INTEGER </span> , since <span class="caps"> INTEGER </span> values are interpreted as indices. </p> <p> You have to adopt the <span class="caps"> GRID </span> _IN part of the INT2LM namelist to match your region. To do this safely proceed as follows: <br/> Perform an <br/> <pre> ncdump -v lon,lat cas1992010100.nc</pre> From the information of the output change the following parameters in <span class="caps"> GRID </span> _IN accordingly: <br/> <code> ie_in_tot, je_in_tot, startlat_in_tot, startlon_in_tot, endlat_in_tot, endlon_in_tot </code> </p>

  @burkhardtrockel in #66dbfa0

<p> Yes, INT2LM can read global data cut out for a sub region. In older INT2LM version ( I am not sure until which version) it was an advantage to cut out the region before running the INT2LM since this would save time because the old versions always took the whole data into account for interpolation. In the versions we currently use the global data are cut in the INT2LM program first before the interpolation is done. <br/> If you have an account at <span class="caps"> DKRZ </span> you can do the cutting there, otherwise you need to transfer the data to your computer first. You may do the latter in a script month by month in order to keep the amount of disk space low at your institute. There is no automatic cutting procedure at <span class="caps"> DKRZ </span> presently. <br/> In oder to save space you can proceed as follows (assume for example your the original file is caf1992010100.nc): </p> <p> Use cdo or ncks to cut out the region you need. Actually cut out a bit larger region since the interpolation needs additional grid points at the boundaries. <br/> For a region with lower left (longitude1, latitude1) and upper right (longitude2, latitude2) perform either <br/> <pre> cdo sellonlatbox,longitude1,longitude2,latitude1,latitude2 caf1992010100.nc cas1992010100.nc ncks -h -d lon,longitude1,longitude2 -d lat,latitude1,latitude2 caf1992010100.nc cas1992010100.nc</pre> In case you use ncks: The coordinates should be <span class="caps"> REAL </span> values not <span class="caps"> INTEGER </span> , since <span class="caps"> INTEGER </span> values are interpreted as indices. </p> <p> You have to adopt the <span class="caps"> GRID </span> _IN part of the INT2LM namelist to match your region. To do this safely proceed as follows: <br/> Perform an <br/> <pre> ncdump -v lon,lat cas1992010100.nc</pre> From the information of the output change the following parameters in <span class="caps"> GRID </span> _IN accordingly: <br/> <code> ie_in_tot, je_in_tot, startlat_in_tot, startlon_in_tot, endlat_in_tot, endlon_in_tot </code> </p>
BR replied

Yes, INT2LM can read global data cut out for a sub region. In older INT2LM version ( I am not sure until which version) it was an advantage to cut out the region before running the INT2LM since this would save time because the old versions always took the whole data into account for interpolation. In the versions we currently use the global data are cut in the INT2LM program first before the interpolation is done.
If you have an account at DKRZ you can do the cutting there, otherwise you need to transfer the data to your computer first. You may do the latter in a script month by month in order to keep the amount of disk space low at your institute. There is no automatic cutting procedure at DKRZ presently.
In oder to save space you can proceed as follows (assume for example your the original file is caf1992010100.nc):

Use cdo or ncks to cut out the region you need. Actually cut out a bit larger region since the interpolation needs additional grid points at the boundaries.
For a region with lower left (longitude1, latitude1) and upper right (longitude2, latitude2) perform either 

cdo sellonlatbox,longitude1,longitude2,latitude1,latitude2 caf1992010100.nc cas1992010100.nc
ncks -h -d lon,longitude1,longitude2 -d lat,latitude1,latitude2 caf1992010100.nc cas1992010100.nc
In case you use ncks: The coordinates should be REAL values not INTEGER , since INTEGER values are interpreted as indices.

You have to adopt the GRID _IN part of the INT2LM namelist to match your region. To do this safely proceed as follows:
Perform an 

ncdump -v lon,lat cas1992010100.nc
From the information of the output change the following parameters in GRID _IN accordingly:
ie_in_tot, je_in_tot, startlat_in_tot, startlon_in_tot, endlat_in_tot, endlon_in_tot

View in channel