Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error plotting an stars_proxy object after st_apply #348

Closed
alexyshr opened this issue Nov 5, 2020 · 3 comments
Closed

Error plotting an stars_proxy object after st_apply #348

alexyshr opened this issue Nov 5, 2020 · 3 comments

Comments

@alexyshr
Copy link

alexyshr commented Nov 5, 2020

I have to convert from stars_proxy to stars to be able to plot it!

Data can be downloaded from here here!

Sys.setenv(GDAL_MAX_BAND_COUNT=1200000)
library(stars)
#> Loading required package: abind
#> Loading required package: sf
#> Linking to GEOS 3.8.0, GDAL 3.0.4, PROJ 6.3.1
path= "./"
ncname = "download5Dcolombia"
ncfile5d = paste0(path, ncname, ".nc")
(st_col5d = read_stars(ncfile5d, proxy=TRUE))
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: Recode from UTF-8 to CP_ACP failed with
#> the error: "Invalid argument".
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: The dataset has several variables
#> that could be identified as vector fields, but not all share the same primary
#> dimension. Consequently they will be ignored.
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: dimension #1 (number) is not a Time
#> dimension.
#> stars_proxy object with 1 attribute in file:
#> $download5Dcolombia.nc
#> [1] "[...]/download5Dcolombia.nc"
#> 
#> dimension(s):
#>        from  to         offset   delta  refsys point
#> x         1  25         -79.35     0.5      NA    NA
#> y         1  35          12.75    -0.5      NA    NA
#> level     1  37             NA      NA udunits    NA
#> number    1  10              0       1      NA    NA
#> time      1 480 2020-01-01 UTC 3 hours POSIXct    NA
#>                                               values x/y
#> x                                               NULL [x]
#> y                                               NULL [y]
#> level  [1,2) [millibars],...,[1000,1025) [millibars]    
#> number                                          NULL    
#> time                                            NULL
#Calculate the cell mean considering all the dimensions
st_col5d_mean = st_apply(st_col5d, c("x", "y"), mean)
#Should this work?
plot(st_col5d_mean)
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: Recode from UTF-8 to CP_ACP failed with
#> the error: "Invalid argument".
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: The dataset has several variables
#> that could be identified as vector fields, but not all share the same primary
#> dimension. Consequently they will be ignored.
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: dimension #1 (number) is not a Time
#> dimension.

#Then, remove the proxy class to be able to plot it
st_col5d_mean = st_as_stars(st_col5d_mean)
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: Recode from UTF-8 to CP_ACP failed with
#> the error: "Invalid argument".
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: The dataset has several variables
#> that could be identified as vector fields, but not all share the same primary
#> dimension. Consequently they will be ignored.
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: dimension #1 (number) is not a Time
#> dimension.
plot(st_col5d_mean)

Session info 
devtools::session_info()
#> - Session info ---------------------------------------------------------------
#>  setting  value                       
#>  version  R version 4.0.2 (2020-06-22)
#>  os       Windows 10 x64              
#>  system   x86_64, mingw32             
#>  ui       RTerm                       
#>  language (EN)                        
#>  collate  English_United States.1252  
#>  ctype    English_United States.1252  
#>  tz       Europe/Berlin               
#>  date     2020-11-05                  
#> 
#> - Packages -------------------------------------------------------------------
#>  package     * version date       lib source                          
#>  abind       * 1.4-5   2016-07-21 [1] CRAN (R 4.0.0)                  
#>  assertthat    0.2.1   2019-03-21 [1] CRAN (R 4.0.2)                  
#>  backports     1.1.10  2020-09-15 [1] CRAN (R 4.0.3)                  
#>  callr         3.5.1   2020-10-13 [1] CRAN (R 4.0.3)                  
#>  class         7.3-17  2020-04-26 [2] CRAN (R 4.0.2)                  
#>  classInt      0.4-3   2020-04-07 [1] CRAN (R 4.0.2)                  
#>  cli           2.1.0   2020-10-12 [1] CRAN (R 4.0.3)                  
#>  crayon        1.3.4   2017-09-16 [1] CRAN (R 4.0.2)                  
#>  curl          4.3     2019-12-02 [1] CRAN (R 4.0.2)                  
#>  DBI           1.1.0   2019-12-15 [1] CRAN (R 4.0.2)                  
#>  desc          1.2.0   2018-05-01 [1] CRAN (R 4.0.2)                  
#>  devtools      2.3.2   2020-09-18 [1] CRAN (R 4.0.2)                  
#>  digest        0.6.27  2020-10-24 [1] CRAN (R 4.0.3)                  
#>  dplyr         1.0.2   2020-08-18 [1] CRAN (R 4.0.2)                  
#>  e1071         1.7-4   2020-10-14 [1] CRAN (R 4.0.3)                  
#>  ellipsis      0.3.1   2020-05-15 [1] CRAN (R 4.0.2)                  
#>  evaluate      0.14    2019-05-28 [1] CRAN (R 4.0.2)                  
#>  fansi         0.4.1   2020-01-08 [1] CRAN (R 4.0.2)                  
#>  fs            1.5.0   2020-07-31 [1] CRAN (R 4.0.2)                  
#>  generics      0.0.2   2018-11-29 [1] CRAN (R 4.0.2)                  
#>  glue          1.4.2   2020-08-27 [1] CRAN (R 4.0.2)                  
#>  highr         0.8     2019-03-20 [1] CRAN (R 4.0.2)                  
#>  htmltools     0.5.0   2020-06-16 [1] CRAN (R 4.0.2)                  
#>  httr          1.4.2   2020-07-20 [1] CRAN (R 4.0.2)                  
#>  KernSmooth    2.23-17 2020-04-26 [2] CRAN (R 4.0.2)                  
#>  knitr         1.29    2020-06-23 [1] CRAN (R 4.0.2)                  
#>  lifecycle     0.2.0   2020-03-06 [1] CRAN (R 4.0.2)                  
#>  lwgeom        0.2-5   2020-06-12 [1] CRAN (R 4.0.2)                  
#>  magrittr      1.5     2014-11-22 [1] CRAN (R 4.0.2)                  
#>  memoise       1.1.0   2017-04-21 [1] CRAN (R 4.0.2)                  
#>  mime          0.9     2020-02-04 [1] CRAN (R 4.0.0)                  
#>  pillar        1.4.6   2020-07-10 [1] CRAN (R 4.0.2)                  
#>  pkgbuild      1.1.0   2020-07-13 [1] CRAN (R 4.0.2)                  
#>  pkgconfig     2.0.3   2019-09-22 [1] CRAN (R 4.0.2)                  
#>  pkgload       1.1.0   2020-05-29 [1] CRAN (R 4.0.2)                  
#>  prettyunits   1.1.1   2020-01-24 [1] CRAN (R 4.0.2)                  
#>  processx      3.4.4   2020-09-03 [1] CRAN (R 4.0.2)                  
#>  ps            1.4.0   2020-10-07 [1] CRAN (R 4.0.3)                  
#>  purrr         0.3.4   2020-04-17 [1] CRAN (R 4.0.2)                  
#>  R6            2.4.1   2019-11-12 [1] CRAN (R 4.0.2)                  
#>  Rcpp          1.0.5   2020-07-06 [1] CRAN (R 4.0.2)                  
#>  remotes       2.2.0   2020-07-21 [1] CRAN (R 4.0.2)                  
#>  rlang         0.4.8   2020-10-08 [1] CRAN (R 4.0.3)                  
#>  rmarkdown     2.3     2020-06-18 [1] CRAN (R 4.0.2)                  
#>  rprojroot     1.3-2   2018-01-03 [1] CRAN (R 4.0.2)                  
#>  sessioninfo   1.1.1   2018-11-05 [1] CRAN (R 4.0.2)                  
#>  sf          * 0.9-7   2020-11-04 [1] Github (r-spatial/sf@92959fd)   
#>  stars       * 0.4-4   2020-11-04 [1] Github (r-spatial/stars@4eaaa76)
#>  stringi       1.5.3   2020-09-09 [1] CRAN (R 4.0.3)                  
#>  stringr       1.4.0   2019-02-10 [1] CRAN (R 4.0.2)                  
#>  testthat      2.3.2   2020-03-02 [1] CRAN (R 4.0.2)                  
#>  tibble        3.0.4   2020-10-12 [1] CRAN (R 4.0.3)                  
#>  tidyselect    1.1.0   2020-05-11 [1] CRAN (R 4.0.2)                  
#>  units         0.6-7   2020-06-13 [1] CRAN (R 4.0.2)                  
#>  usethis       1.6.3   2020-09-17 [1] CRAN (R 4.0.2)                  
#>  vctrs         0.3.4   2020-08-29 [1] CRAN (R 4.0.2)                  
#>  withr         2.3.0   2020-09-22 [1] CRAN (R 4.0.3)                  
#>  xfun          0.18    2020-09-29 [1] CRAN (R 4.0.3)                  
#>  xml2          1.3.2   2020-04-23 [1] CRAN (R 4.0.2)                  
#>  yaml          2.2.1   2020-02-01 [1] CRAN (R 4.0.2)                  
#> 
#> [1] E:/Documents/R/win-library/4.0
#> [2] C:/Program Files/R/R-4.0.2/library
@edzer
Copy link
Member

edzer commented Nov 5, 2020

Wow, FIVE dimensions!! The auto computation of downsample fails here, and e.g.

plot(st_col5d, downsample=c(1,1,10,10,10))

gives something useful, as does

plot(st_col5d_mean, downsample=0)

Thanks, great issue!
@alexyshr
Copy link
Author

alexyshr commented Nov 7, 2020

Thank you!.

So, to plot a stars_proxy object, it is necessary to use the downsample parameter. It is stated on plot.stars_proxy help: when plotting a subsetted stars_proxy object, the default value for argument downsample will not be computed correctly, and it and has to be set manually.

A huge advantage here: we don't need to change the maximum number of bands available for GDAL processes (variable GDAL_MAX_BAND_COUNT) while working with stars_proxy.

library(stars)
#> Loading required package: abind
#> Loading required package: sf
#> Linking to GEOS 3.8.0, GDAL 3.0.4, PROJ 6.3.1
path= "./"
ncname = "download5Dcolombia"
ncfile5d = paste0(path, ncname, ".nc")
(st_col5d = read_stars(ncfile5d, proxy=TRUE))
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: Recode from UTF-8 to CP_ACP failed with
#> the error: "Invalid argument".
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: The dataset has several variables
#> that could be identified as vector fields, but not all share the same primary
#> dimension. Consequently they will be ignored.
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: dimension #1 (number) is not a Time
#> dimension.
#> stars_proxy object with 1 attribute in file:
#> $download5Dcolombia.nc
#> [1] "[...]/download5Dcolombia.nc"
#> 
#> dimension(s):
#>        from  to         offset   delta  refsys point
#> x         1  25         -79.35     0.5      NA    NA
#> y         1  35          12.75    -0.5      NA    NA
#> level     1  37             NA      NA udunits    NA
#> number    1  10              0       1      NA    NA
#> time      1 480 2020-01-01 UTC 3 hours POSIXct    NA
#>                                               values x/y
#> x                                               NULL [x]
#> y                                               NULL [y]
#> level  [1,2) [millibars],...,[1000,1025) [millibars]    
#> number                                          NULL    
#> time                                            NULL
#Calculate the cell mean considering all the dimensions
st_col5d_mean = st_apply(st_col5d, c("x", "y"), mean)
#To plot stars_proxy, use downsample parameter (see plot.stars_proxy help):
#when plotting a subsetted stars_proxy object, 
#the default value for argument downsample will not be computed correctly, 
#and it has to be set manually.
plot(st_col5d_mean, downsample=c(1,1,10,10,10))
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: Recode from UTF-8 to CP_ACP failed with
#> the error: "Invalid argument".
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: The dataset has several variables
#> that could be identified as vector fields, but not all share the same primary
#> dimension. Consequently they will be ignored.
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: dimension #1 (number) is not a Time
#> dimension.

#downsample=0, no downsample takes place
plot(st_col5d_mean, downsample=0)
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: Recode from UTF-8 to CP_ACP failed with
#> the error: "Invalid argument".
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: The dataset has several variables
#> that could be identified as vector fields, but not all share the same primary
#> dimension. Consequently they will be ignored.
#> Warning in CPL_read_gdal(as.character(x), as.character(options),
#> as.character(driver), : GDAL Message 1: dimension #1 (number) is not a Time
#> dimension.

Session info 
devtools::session_info()
#> - Session info ---------------------------------------------------------------
#>  setting  value                       
#>  version  R version 4.0.2 (2020-06-22)
#>  os       Windows 10 x64              
#>  system   x86_64, mingw32             
#>  ui       RTerm                       
#>  language (EN)                        
#>  collate  English_United States.1252  
#>  ctype    English_United States.1252  
#>  tz       Europe/Berlin               
#>  date     2020-11-07                  
#> 
#> - Packages -------------------------------------------------------------------
#>  package     * version date       lib source                          
#>  abind       * 1.4-5   2016-07-21 [1] CRAN (R 4.0.0)                  
#>  assertthat    0.2.1   2019-03-21 [1] CRAN (R 4.0.2)                  
#>  backports     1.1.10  2020-09-15 [1] CRAN (R 4.0.3)                  
#>  callr         3.5.1   2020-10-13 [1] CRAN (R 4.0.3)                  
#>  class         7.3-17  2020-04-26 [2] CRAN (R 4.0.2)                  
#>  classInt      0.4-3   2020-04-07 [1] CRAN (R 4.0.2)                  
#>  cli           2.1.0   2020-10-12 [1] CRAN (R 4.0.3)                  
#>  crayon        1.3.4   2017-09-16 [1] CRAN (R 4.0.2)                  
#>  curl          4.3     2019-12-02 [1] CRAN (R 4.0.2)                  
#>  DBI           1.1.0   2019-12-15 [1] CRAN (R 4.0.2)                  
#>  desc          1.2.0   2018-05-01 [1] CRAN (R 4.0.2)                  
#>  devtools      2.3.2   2020-09-18 [1] CRAN (R 4.0.2)                  
#>  digest        0.6.27  2020-10-24 [1] CRAN (R 4.0.3)                  
#>  dplyr         1.0.2   2020-08-18 [1] CRAN (R 4.0.2)                  
#>  e1071         1.7-4   2020-10-14 [1] CRAN (R 4.0.3)                  
#>  ellipsis      0.3.1   2020-05-15 [1] CRAN (R 4.0.2)                  
#>  evaluate      0.14    2019-05-28 [1] CRAN (R 4.0.2)                  
#>  fansi         0.4.1   2020-01-08 [1] CRAN (R 4.0.2)                  
#>  fs            1.5.0   2020-07-31 [1] CRAN (R 4.0.2)                  
#>  generics      0.0.2   2018-11-29 [1] CRAN (R 4.0.2)                  
#>  glue          1.4.2   2020-08-27 [1] CRAN (R 4.0.2)                  
#>  highr         0.8     2019-03-20 [1] CRAN (R 4.0.2)                  
#>  htmltools     0.5.0   2020-06-16 [1] CRAN (R 4.0.2)                  
#>  httr          1.4.2   2020-07-20 [1] CRAN (R 4.0.2)                  
#>  KernSmooth    2.23-17 2020-04-26 [2] CRAN (R 4.0.2)                  
#>  knitr         1.29    2020-06-23 [1] CRAN (R 4.0.2)                  
#>  lifecycle     0.2.0   2020-03-06 [1] CRAN (R 4.0.2)                  
#>  lwgeom        0.2-5   2020-06-12 [1] CRAN (R 4.0.2)                  
#>  magrittr      1.5     2014-11-22 [1] CRAN (R 4.0.2)                  
#>  memoise       1.1.0   2017-04-21 [1] CRAN (R 4.0.2)                  
#>  mime          0.9     2020-02-04 [1] CRAN (R 4.0.0)                  
#>  pillar        1.4.6   2020-07-10 [1] CRAN (R 4.0.2)                  
#>  pkgbuild      1.1.0   2020-07-13 [1] CRAN (R 4.0.2)                  
#>  pkgconfig     2.0.3   2019-09-22 [1] CRAN (R 4.0.2)                  
#>  pkgload       1.1.0   2020-05-29 [1] CRAN (R 4.0.2)                  
#>  prettyunits   1.1.1   2020-01-24 [1] CRAN (R 4.0.2)                  
#>  processx      3.4.4   2020-09-03 [1] CRAN (R 4.0.2)                  
#>  ps            1.4.0   2020-10-07 [1] CRAN (R 4.0.3)                  
#>  purrr         0.3.4   2020-04-17 [1] CRAN (R 4.0.2)                  
#>  R6            2.4.1   2019-11-12 [1] CRAN (R 4.0.2)                  
#>  Rcpp          1.0.5   2020-07-06 [1] CRAN (R 4.0.2)                  
#>  remotes       2.2.0   2020-07-21 [1] CRAN (R 4.0.2)                  
#>  rlang         0.4.8   2020-10-08 [1] CRAN (R 4.0.3)                  
#>  rmarkdown     2.3     2020-06-18 [1] CRAN (R 4.0.2)                  
#>  rprojroot     1.3-2   2018-01-03 [1] CRAN (R 4.0.2)                  
#>  sessioninfo   1.1.1   2018-11-05 [1] CRAN (R 4.0.2)                  
#>  sf          * 0.9-7   2020-11-04 [1] Github (r-spatial/sf@92959fd)   
#>  stars       * 0.4-4   2020-11-04 [1] Github (r-spatial/stars@4eaaa76)
#>  stringi       1.5.3   2020-09-09 [1] CRAN (R 4.0.3)                  
#>  stringr       1.4.0   2019-02-10 [1] CRAN (R 4.0.2)                  
#>  testthat      2.3.2   2020-03-02 [1] CRAN (R 4.0.2)                  
#>  tibble        3.0.4   2020-10-12 [1] CRAN (R 4.0.3)                  
#>  tidyselect    1.1.0   2020-05-11 [1] CRAN (R 4.0.2)                  
#>  units         0.6-7   2020-06-13 [1] CRAN (R 4.0.2)                  
#>  usethis       1.6.3   2020-09-17 [1] CRAN (R 4.0.2)                  
#>  vctrs         0.3.4   2020-08-29 [1] CRAN (R 4.0.2)                  
#>  withr         2.3.0   2020-09-22 [1] CRAN (R 4.0.3)                  
#>  xfun          0.18    2020-09-29 [1] CRAN (R 4.0.3)                  
#>  xml2          1.3.2   2020-04-23 [1] CRAN (R 4.0.2)                  
#>  yaml          2.2.1   2020-02-01 [1] CRAN (R 4.0.2)                  
#> 
#> [1] E:/Documents/R/win-library/4.0
#> [2] C:/Program Files/R/R-4.0.2/library
@edzer edzer mentioned this issue Nov 8, 2020
Open
@edzer edzer closed this as completed in 2781eb9 Apr 28, 2021
@edzer
Copy link
Member

edzer commented Apr 28, 2021

downsample is now printed by plot(), if it is not set by the user (and larger than 1).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
2 participants
@edzer @alexyshr