Vector data compression

If you need to compress vector data that lacks an internal compression mechanism (GeoPackage, Shapefile, FlatGeobuf), consider using the Seek-Optimized ZIP profile implemented in GDAL, which minimizes the overhead associated with loading data compared to the standard ZIP format.

Code

library("sf")
library("zip")

set.seed(1)

# generate some data
n = 500000
df = data.frame(x = runif(n, 160000, 877000), y = runif(n, 126000, 780000))
df = st_as_sf(df, coords = c("x", "y"), crs = "EPSG:2180")
buff = st_buffer(df, dist = 5000)

Comparison of disk space usage (MB):

# uncompressed geopackage
write_sf(buff, "test.gpkg", driver = "GPKG")
file.size("test.gpkg") / 1024 ^ 2

## [1] 1005.848

# compressed geopackage with SOZip
write_sf(buff, "test.gpkg.zip", driver = "GPKG")
file.size("test.gpkg.zip") / 1024 ^ 2

## [1] 542.6809

# compressed geopackage without SOZip
zip::zip("test.zip", "test.gpkg")
file.size("test.zip") / 1024 ^ 2

## [1] 541.7384

Comparison of data loading performance:

# uncompressed geopackage
system.time({
  read_sf("test.gpkg")
})

##    user  system elapsed 
##    3.81    1.33    5.16

# compressed geopackage with SOZip
system.time({
  read_sf("test.gpkg.zip")
})

##    user  system elapsed 
##    5.41    1.13    6.53

# compressed geopackage without SOZip
system.time({
  read_sf("/vsizip/test.zip")
})

##    user  system elapsed 
##   19.97    1.55   21.44

# first unzip, then read geopackage
system.time({
  unzip("test.gpkg.zip", exdir = "test")
  read_sf("test/test.gpkg")
})

##    user  system elapsed 
##    7.31    1.92    9.20