Downloading and preprocessing EDGAR dataset¶
GCP-CH4 uses EDGAR CH4 fluxes inventory as for its inversions’ anthropogenic fluxes a priori. This inventory can be downloaded from the EDGAR webiste.
The script bellow is an example for downloading and unzipping the EDGARv6.0 CH4 dataset:
Show/Hide download_EDGARv6.0.sh
1#!/usr/bin/bash
2
3DIR="$(cd $(dirname $0); pwd)"
4
5year_start=2000
6year_stop=2018
7
8base_url="https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/EDGAR/datasets/v60_GHG/CH4"
9
10sector_lists=( \
11 'AGS' 'AWB' 'CHE' 'ENE' 'ENF' 'FFF' 'IND' 'IRO' 'MNM' \
12 'PRO_COAL' 'PRO_GAS' 'PRO_OIL' 'RCO' 'REF_TRF' 'SWD_INC' 'SWD_LDF' \
13 'TNR_Aviation_CDS' 'TNR_Aviation_CRS' 'TNR_Aviation_LTO' 'TNR_Aviation_SPS' \
14 'TNR_Other' 'TNR_Ship' 'TRO_noRES' 'WWT' \
15)
16
17for sector in "${sector_list[@]}"; do
18 echo "Downloading Sector ${sector}"
19 mkdir -p "${DIR}/${sector}"
20
21 if [ "${sector}" = "TNR_Aviation_SPS" ]; then
22 if [ $year_stop -gt 2003 ];then
23 sector_year_stop=2003
24 fi
25 else
26 sector_year_stop=$year_stop
27 fi
28
29 for year in $(seq ${year_start} ${sector_year_stop}); do
30 zip_file="${DIR}/${sector}/CH4_${sector}_${year}.zip"
31 wget "${base_url}/${sector}/v6.0_CH4_${year}_monthly_${sector}_nc.zip" \
32 --output-document "${zip_file}"
33 unzip -o "${zip_file}" -d "${DIR}/${sector}"
34 rm -vf "${zip_file}"
35
36 sleep 1 # Can prevent "connection denied" errors
37 fi
38 done
39done
EDGAR fluxes are divided in numerous sectors and use a \(0.1° \times 0.1°\) grid. GCP-CH4 a priori fluxes use a coarser grid and fewer sectors. The YAML files below groups and regrids EDGAR’s fluxes files to match the GCP-CH4 a priori format.
With those example the files are written in the folder /tmp/GCP_Fluxes
.
Warning
The YAML file gcp-regrid-and-group-edgar-full.yml
is not automatically
tested, but the second example file gcp-regrid-and-group-edgar.yml
is.
Note
The “TNR_Aviation_SPS” sector ends at the end of year 2003, to compute the
fluxes between 2004 and 2018 remove the two appearances of this sector in
the YAML file (one in controlvect/transfrom_pipe/sum_trans
and one in
datavect/components/flux/parameters
) and modify the datei
and
datef
accordingly.
Show/Hide YAML file gcp-regrid-and-group-edgar-full.yml
(not automatically tested)
1#####################
2# PYCIF config file #
3#####################
4
5###############################################################################
6#
7# Regrid and sum by categories EDGARv6.0 CH4 monthly fluxe files
8#
9###############################################################################
10# Paths
11
12# Outputs directory
13rootdir: &rootdir /tmp/cif-gcp
14
15# EDGAR dataset directory
16edgardir: &edgardir /path/to/edgar/download/directory
17
18# Target domain file directory
19domaindir: &domaindir /path/to/domain/directory
20
21# Target domain filename
22domainfile: &domainfile domain_1x1.nc
23
24###############################################################################
25# PYCIF parameters
26
27verbose: 1
28
29outdir: !join &outdir [*rootdir, /GCP_Fluxes]
30workdir: !join [*rootdir, /workdir]
31logfile: pycif.log
32
33datei: 2000-01-01
34datef: 2004-01-01
35
36###############################################################################
37
38platform:
39 plugin:
40 name: docker
41 version: cif
42
43###############################################################################
44
45obsoperator:
46 plugin:
47 name: standard
48 version: std
49
50 init_inputs:
51 components:
52 flux: []
53 ignore_model: true
54
55###############################################################################
56
57obsvect:
58 plugin:
59 name: standard
60 version: std
61
62###############################################################################
63
64domain:
65 plugin:
66 name: gridded_netcdf
67 version: std
68 dir: *domaindir
69 file: *domainfile
70
71###############################################################################
72
73controlvect:
74 plugin:
75 name: standard
76 version: std
77
78 transform_pipe:
79 sum_live:
80 plugin:
81 name: families
82 version: std
83 type: transform
84 component: flux
85 parameters_in:
86 - ENF
87 - MNM
88 parameter_out: live
89
90 sum_waste:
91 plugin:
92 name: families
93 version: std
94 type: transform
95 component: flux
96 parameters_in:
97 - SWD_LDF
98 - SWD_INC
99 - WWT
100 parameter_out: waste
101
102 sum_oilgas:
103 plugin:
104 name: families
105 version: std
106 type: transform
107 component: flux
108 parameters_in:
109 - PRO_OIL
110 - PRO_GAS
111 - FFF
112 parameter_out: oilgas
113
114 sum_indus:
115 plugin:
116 name: families
117 version: std
118 type: transform
119 component: flux
120 parameters_in:
121 - CHE
122 - IRO
123 - ENE
124 - IND
125 - REF_TRF
126 parameter_out: indus
127
128 sum_trans:
129 plugin:
130 name: families
131 version: std
132 type: transform
133 component: flux
134 parameters_in:
135 - TRO_noRES
136 - TNR_Aviation_CDS
137 - TNR_Aviation_CRS
138 - TNR_Aviation_LTO
139 - TNR_Aviation_SPS
140 - TNR_Other
141 - TNR_Ship
142 parameter_out: trans
143
144 dump_live:
145 plugin:
146 name: dump2format
147 version: std
148 type: transform
149 component: flux
150 parameter: live
151 dump_dir: !join [*outdir, /live]
152 dump_file: fch4.live.gcp2022.1x1.%Y.nc
153 dump_format:
154 type: flux
155 name: gridded_netcdf
156 version: std
157 use_defined_domain: True
158 resample:
159 file_freq: 1YS
160 var_freq: 1MS
161
162 dump_rice:
163 plugin:
164 name: dump2format
165 version: std
166 type: transform
167 component: flux
168 parameter: AGS
169 dump_dir: !join [*outdir, /rice]
170 dump_file: fch4.rice.gcp2022.1x1.%Y.nc
171 dump_format:
172 type: flux
173 name: gridded_netcdf
174 version: std
175 use_defined_domain: True
176 resample:
177 file_freq: 1YS
178 var_freq: 1MS
179
180 dump_waste:
181 plugin:
182 name: dump2format
183 version: std
184 type: transform
185 component: flux
186 parameter: waste
187 dump_dir: !join [*outdir, /waste]
188 dump_file: fch4.waste.gcp2022.1x1.%Y.nc
189 dump_format:
190 type: flux
191 name: gridded_netcdf
192 version: std
193 use_defined_domain: True
194 resample:
195 file_freq: 1YS
196 var_freq: 1MS
197
198 dump_oilgas:
199 plugin:
200 name: dump2format
201 version: std
202 type: transform
203 component: flux
204 parameter: oilgas
205 dump_dir: !join [*outdir, /oilgas]
206 dump_file: fch4.oilgas.gcp2022.1x1.%Y.nc
207 dump_format:
208 type: flux
209 name: gridded_netcdf
210 version: std
211 use_defined_domain: True
212 resample:
213 file_freq: 1YS
214 var_freq: 1MS
215
216 dump_coal:
217 plugin:
218 name: dump2format
219 version: std
220 type: transform
221 component: flux
222 parameter: PRO_COAL
223 dump_dir: !join [*outdir, /coal]
224 dump_file: fch4.coal.gcp2022.1x1.%Y.nc
225 dump_format:
226 type: flux
227 name: gridded_netcdf
228 version: std
229 use_defined_domain: True
230 resample:
231 file_freq: 1YS
232 var_freq: 1MS
233
234 dump_indus:
235 plugin:
236 name: dump2format
237 version: std
238 type: transform
239 component: flux
240 parameter: indus
241 dump_dir: !join [*outdir, /indus]
242 dump_file: fch4.indus.gcp2022.1x1.%Y.nc
243 dump_format:
244 type: flux
245 name: gridded_netcdf
246 version: std
247 use_defined_domain: True
248 resample:
249 file_freq: 1YS
250 var_freq: 1MS
251
252 dump_trans:
253 plugin:
254 name: dump2format
255 version: std
256 type: transform
257 component: flux
258 parameter: trans
259 dump_dir: !join [*outdir, /trans]
260 dump_file: fch4.trans.gcp2022.1x1.%Y.nc
261 dump_format:
262 type: flux
263 name: gridded_netcdf
264 version: std
265 use_defined_domain: True
266 resample:
267 file_freq: 1YS
268 var_freq: 1MS
269
270 dump_biof:
271 plugin:
272 name: dump2format
273 version: std
274 type: transform
275 component: flux
276 parameter: RCO
277 dump_dir: !join [*outdir, /biof]
278 dump_file: fch4.biof.gcp2022.1x1.%Y.nc
279 dump_format:
280 type: flux
281 name: gridded_netcdf
282 version: std
283 use_defined_domain: True
284 resample:
285 file_freq: 1YS
286 var_freq: 1MS
287
288
289###############################################################################
290
291datavect:
292 plugin:
293 name: standard
294 version: std
295
296 components:
297 flux:
298 dir: *edgardir
299 file_freq: 1MS
300 varname: emi_ch4
301
302 regrid:
303 method: mass-conservation
304
305 parameters:
306 AGS:
307 plugin:
308 name: gridded_netcdf
309 version: std
310 type: flux
311 file: AGS/v6.0_CH4_%Y_%-m_AGS.0.1x0.1.nc
312 add_time_coord: True
313 var_freq: 1M
314 lon_min: 0.0
315 lon_max: 360.0
316 delta_lon: 0.1
317 delta_lat: 0.1
318
319 CHE:
320 plugin:
321 name: gridded_netcdf
322 version: std
323 type: flux
324 file: CHE/v6.0_CH4_%Y_%-m_CHE.0.1x0.1.nc
325 add_time_coord: True
326 var_freq: 1M
327 lon_min: 0.0
328 lon_max: 360.0
329 delta_lon: 0.1
330 delta_lat: 0.1
331
332 ENE:
333 plugin:
334 name: gridded_netcdf
335 version: std
336 type: flux
337 file: ENE/v6.0_CH4_%Y_%-m_ENE.0.1x0.1.nc
338 add_time_coord: True
339 var_freq: 1M
340 lon_min: 0.0
341 lon_max: 360.0
342 delta_lon: 0.1
343 delta_lat: 0.1
344
345 ENF:
346 plugin:
347 name: gridded_netcdf
348 version: std
349 type: flux
350 file: ENF/v6.0_CH4_%Y_%-m_ENF.0.1x0.1.nc
351 add_time_coord: True
352 var_freq: 1M
353 lon_min: 0.0
354 lon_max: 360.0
355 delta_lon: 0.1
356 delta_lat: 0.1
357
358 FFF:
359 plugin:
360 name: gridded_netcdf
361 version: std
362 type: flux
363 file: FFF/v6.0_CH4_%Y_%-m_FFF.0.1x0.1.nc
364 add_time_coord: True
365 var_freq: 1M
366 lon_min: 0.0
367 lon_max: 360.0
368 delta_lon: 0.1
369 delta_lat: 0.1
370
371 IND:
372 plugin:
373 name: gridded_netcdf
374 version: std
375 type: flux
376 file: IND/v6.0_CH4_%Y_%-m_IND.0.1x0.1.nc
377 add_time_coord: True
378 var_freq: 1M
379 lon_min: 0.0
380 lon_max: 360.0
381 delta_lon: 0.1
382 delta_lat: 0.1
383
384 IRO:
385 plugin:
386 name: gridded_netcdf
387 version: std
388 type: flux
389 file: IRO/v6.0_CH4_%Y_%-m_IRO.0.1x0.1.nc
390 add_time_coord: True
391 var_freq: 1M
392 lon_min: 0.0
393 lon_max: 360.0
394 delta_lon: 0.1
395 delta_lat: 0.1
396
397 MNM:
398 plugin:
399 name: gridded_netcdf
400 version: std
401 type: flux
402 file: MNM/v6.0_CH4_%Y_%-m_MNM.0.1x0.1.nc
403 add_time_coord: True
404 var_freq: 1M
405 lon_min: 0.0
406 lon_max: 360.0
407 delta_lon: 0.1
408 delta_lat: 0.1
409
410 PRO_COAL:
411 plugin:
412 name: gridded_netcdf
413 version: std
414 type: flux
415 file: PRO_COAL/v6.0_CH4_%Y_%-m_PRO_COAL.0.1x0.1.nc
416 add_time_coord: True
417 var_freq: 1M
418 lon_min: 0.0
419 lon_max: 360.0
420 delta_lon: 0.1
421 delta_lat: 0.1
422
423 PRO_GAS:
424 plugin:
425 name: gridded_netcdf
426 version: std
427 type: flux
428 file: PRO_GAS/v6.0_CH4_%Y_%-m_PRO_GAS.0.1x0.1.nc
429 add_time_coord: True
430 var_freq: 1M
431 lon_min: 0.0
432 lon_max: 360.0
433 delta_lon: 0.1
434 delta_lat: 0.1
435
436 PRO_OIL:
437 plugin:
438 name: gridded_netcdf
439 version: std
440 type: flux
441 file: PRO_OIL/v6.0_CH4_%Y_%-m_PRO_OIL.0.1x0.1.nc
442 add_time_coord: True
443 var_freq: 1M
444 lon_min: 0.0
445 lon_max: 360.0
446 delta_lon: 0.1
447 delta_lat: 0.1
448
449 RCO:
450 plugin:
451 name: gridded_netcdf
452 version: std
453 type: flux
454 file: RCO/v6.0_CH4_%Y_%-m_RCO.0.1x0.1.nc
455 add_time_coord: True
456 var_freq: 1M
457 lon_min: 0.0
458 lon_max: 360.0
459 delta_lon: 0.1
460 delta_lat: 0.1
461
462 REF_TRF:
463 plugin:
464 name: gridded_netcdf
465 version: std
466 type: flux
467 file: REF_TRF/v6.0_CH4_%Y_%-m_REF_TRF.0.1x0.1.nc
468 add_time_coord: True
469 var_freq: 1M
470 lon_min: 0.0
471 lon_max: 360.0
472 delta_lon: 0.1
473 delta_lat: 0.1
474
475 SWD_INC:
476 plugin:
477 name: gridded_netcdf
478 version: std
479 type: flux
480 file: SWD_INC/v6.0_CH4_%Y_%-m_SWD_INC.0.1x0.1.nc
481 add_time_coord: True
482 var_freq: 1M
483 lon_min: 0.0
484 lon_max: 360.0
485 delta_lon: 0.1
486 delta_lat: 0.1
487
488 SWD_LDF:
489 plugin:
490 name: gridded_netcdf
491 version: std
492 type: flux
493 file: SWD_LDF/v6.0_CH4_%Y_%-m_SWD_LDF.0.1x0.1.nc
494 add_time_coord: True
495 var_freq: 1M
496 lon_min: 0.0
497 lon_max: 360.0
498 delta_lon: 0.1
499 delta_lat: 0.1
500
501 TNR_Aviation_CDS:
502 plugin:
503 name: gridded_netcdf
504 version: std
505 type: flux
506 file: TNR_Aviation_CDS/v6.0_CH4_%Y_%-m_TNR_Aviation_CDS.0.1x0.1.nc
507 add_time_coord: True
508 var_freq: 1M
509 lon_min: 0.0
510 lon_max: 360.0
511 delta_lon: 0.1
512 delta_lat: 0.1
513
514 TNR_Aviation_CRS:
515 plugin:
516 name: gridded_netcdf
517 version: std
518 type: flux
519 file: TNR_Aviation_CRS/v6.0_CH4_%Y_%-m_TNR_Aviation_CRS.0.1x0.1.nc
520 add_time_coord: True
521 var_freq: 1M
522 lon_min: 0.0
523 lon_max: 360.0
524 delta_lon: 0.1
525 delta_lat: 0.1
526
527 TNR_Aviation_LTO:
528 plugin:
529 name: gridded_netcdf
530 version: std
531 type: flux
532 file: TNR_Aviation_LTO/v6.0_CH4_%Y_%-m_TNR_Aviation_LTO.0.1x0.1.nc
533 add_time_coord: True
534 var_freq: 1M
535 lon_min: 0.0
536 lon_max: 360.0
537 delta_lon: 0.1
538 delta_lat: 0.1
539
540 TNR_Aviation_SPS:
541 plugin:
542 name: gridded_netcdf
543 version: std
544 type: flux
545 file: TNR_Aviation_SPS/v6.0_CH4_%Y_%-m_TNR_Aviation_SPS.0.1x0.1.nc
546 add_time_coord: True
547 var_freq: 1M
548 lon_min: 0.0
549 lon_max: 360.0
550 delta_lon: 0.1
551 delta_lat: 0.1
552
553 TNR_Other:
554 plugin:
555 name: gridded_netcdf
556 version: std
557 type: flux
558 file: TNR_Other/v6.0_CH4_%Y_%-m_TNR_Other.0.1x0.1.nc
559 add_time_coord: True
560 var_freq: 1M
561 lon_min: 0.0
562 lon_max: 360.0
563 delta_lon: 0.1
564 delta_lat: 0.1
565
566 TNR_Ship:
567 plugin:
568 name: gridded_netcdf
569 version: std
570 type: flux
571 file: TNR_Ship/v6.0_CH4_%Y_%-m_TNR_Ship.0.1x0.1.nc
572 add_time_coord: True
573 var_freq: 1M
574 lon_min: 0.0
575 lon_max: 360.0
576 delta_lon: 0.1
577 delta_lat: 0.1
578
579 TRO_noRES:
580 plugin:
581 name: gridded_netcdf
582 version: std
583 type: flux
584 file: TRO_noRES/v6.0_CH4_%Y_%-m_TRO_noRES.0.1x0.1.nc
585 add_time_coord: True
586 var_freq: 1M
587 lon_min: 0.0
588 lon_max: 360.0
589 delta_lon: 0.1
590 delta_lat: 0.1
591
592 WWT:
593 plugin:
594 name: gridded_netcdf
595 version: std
596 type: flux
597 file: WWT/v6.0_CH4_%Y_%-m_WWT.0.1x0.1.nc
598 add_time_coord: True
599 var_freq: 1M
600 lon_min: 0.0
601 lon_max: 360.0
602 delta_lon: 0.1
603 delta_lat: 0.1
Show/Hide YAML file gcp-regrid-and-group-edgar.yml
(automatically tested)
TODO !!
The groups are:
livestocks
ENF
MNM
rice
AGS
waste
SWD_LDF
SWD_INC
WWT
oil & gas
PRO_OIL
PRO_GAS
FFF
coal
PRO_COAL
industry
CHE
IRO
ENE
IND
REF_TRF
transports
TRO_noRES
TNR_Aviation_CDS
TNR_Aviation_CRS
TNR_Aviation_LTO
TNR_Aviation_SPS
TNR_Other
TNR_Ship
Here is the target domain NetCDF file which contains the coordinates for a \(1° \times 1°\) grid.
Show/Hide domain_1x1.nc
netcdf domain_1x1 {
dimensions:
lat = 180 ;
lon = 360 ;
bounds = 2 ;
variables:
double lat(lat) ;
lat:_FillValue = NaN ;
lat:long_name = "latitude" ;
lat:standard_name = "latitude" ;
lat:units = "degrees_north" ;
lat:bounds = "lat_bounds" ;
double lon(lon) ;
lon:_FillValue = NaN ;
lon:long_name = "longitude" ;
lon:standard_name = "longitude" ;
lon:units = "degrees_east" ;
lon:bounds = "lon_bounds" ;
double lat_bounds(lat, bounds) ;
lat_bounds:_FillValue = NaN ;
lat_bounds:long_name = "latitude bounds" ;
lat_bounds:standard_name = "latitude_bounds" ;
double lon_bounds(lon, bounds) ;
lon_bounds:_FillValue = NaN ;
lon_bounds:long_name = "longitude bounds" ;
lon_bounds:standard_name = "longitude_bounds" ;
// global attributes:
:coordinates = "lon_bounds lat_bounds" ;
}