| Title: | Territorial data acquisition |
|---|---|
| Description: | **DATALIFT** (Territorial Data Acquisition) is an R package designed to facilitate the acquisition, download and preparation of spatial and environmental datasets from multiple online data providers. It provides a unified interface to access and process data related to topography, land cover, soils, meteorology, LiDAR and satellite imagery. DATALIFT integrates several open-source tools and packages, including 'happign', 'lidarHD' and 'RADIS', to support reproducible territorial data acquisition workflows. |
| Authors: | Fabrice Vinatier [cre, aut], Sébastien Troiano [aut] |
| Maintainer: | Fabrice Vinatier <[email protected]> |
| License: | GPL (>= 3) |
| Version: | 0.1.0 |
| Built: | 2026-11-13 16:43:00 UTC |
| Source: | https://forge.inrae.fr/fabrice.vinatier/datalift.git |
Express a file size in the most appropriate binary unit, from bytes to petabytes.
bytes_to_human(bytes, digits = 1)bytes_to_human(bytes, digits = 1)
bytes |
Numeric scalar. Non-negative file size in bytes. |
digits |
Integer. Number of decimal places used for rounding. |
Character string containing the rounded value and its unit ('B', 'Ko', 'Mo', 'Go', 'To' or 'Po').
[folder_size()]
bytes_to_human(1536) bytes_to_human(1048576, digits = 2)bytes_to_human(1536) bytes_to_human(1048576, digits = 2)
Run the loader associated with a dataset name, measure its elapsed execution time and calculate the size of the corresponding output folder.
calc_duration_size(dataset = "MNE", vWIN, path_export, delete_files = TRUE)calc_duration_size(dataset = "MNE", vWIN, path_export, delete_files = TRUE)
dataset |
Character string identifying the dataset and corresponding loader function. For example, '"MNE"' calls 'load_MNE()'. |
vWIN |
Spatial object passed to the loader as the study area. |
path_export |
Character string. Root output directory passed to the loader. |
delete_files |
Logical. If 'TRUE', remove 'path_export' recursively after the benchmark. Defaults to 'TRUE'. |
The loader is resolved by constructing the function name 'load_<dataset>'. The output size is measured in the subdirectory named after 'dataset'. Use a dedicated temporary output directory when 'delete_files = TRUE'.
Numeric vector of length two: elapsed time in seconds and output size in bytes.
## Not run: area <- sf::st_read("study_area.gpkg") calc_duration_size("MNE", area, tempfile()) ## End(Not run)## Not run: area <- sf::st_read("study_area.gpkg") calc_duration_size("MNE", area, tempfile()) ## End(Not run)
Divide a spatial bounding box into tiles and download each tile from a WMS service at a fixed map resolution. Each tile is exported as JPEG2000.
export_wms_from_sf_fixed_res( sf_obj, layer, crs_out = 2154, resolution = 0.05, max_pixels = 5000, outdir = "tiles" )export_wms_from_sf_fixed_res( sf_obj, layer, crs_out = 2154, resolution = 0.05, max_pixels = 5000, outdir = "tiles" )
sf_obj |
An 'sf' or 'sfc' spatial object defining the export extent. |
layer |
Character string. WMS layer identifier understood by the configured service. |
crs_out |
Coordinate reference system used to interpret the bounding box. Defaults to EPSG:2154. |
resolution |
Numeric. Requested raster resolution in units of 'crs_out'. Defaults to 0.05. |
max_pixels |
Integer. Maximum number of pixels along each tile dimension. Defaults to 5000. |
outdir |
Character string. Output directory for the JPEG2000 tiles. |
The maximum nominal tile width and height are each 'max_pixels * resolution' map units. The output uses the GDAL 'JP2OpenJPEG' driver with lossless encoding options ('QUALITY=100' and 'REVERSIBLE=YES'). Availability of that GDAL driver is required.
The function writes files named 'tile_<n>.jp2' to 'outdir' and returns no consolidated raster object.
[load_PCRS()]
## Not run: area <- sf::st_read("study_area.gpkg") export_wms_from_sf_fixed_res( area, layer = "PCRS.LAMB93", resolution = 0.05, outdir = "OUT/PCRS" ) ## End(Not run)## Not run: area <- sf::st_read("study_area.gpkg") export_wms_from_sf_fixed_res( area, layer = "PCRS.LAMB93", resolution = 0.05, outdir = "OUT/PCRS" ) ## End(Not run)
Authenticate with the Météo-France API, submit observation-data requests for successive calendar years and combine the returned tables.
fct.read.meteofrance( auth_base64 = Sys.getenv("METEOFRANCE_AUTH_BASE64"), interval = "daily", station_id = "34237001", from_date = "2000-01-01", to_date = "2010-12-31" )fct.read.meteofrance( auth_base64 = Sys.getenv("METEOFRANCE_AUTH_BASE64"), interval = "daily", station_id = "34237001", from_date = "2000-01-01", to_date = "2010-12-31" )
auth_base64 |
Character string. Base64-encoded API credentials. By default, read from the 'METEOFRANCE_AUTH_BASE64' environment variable. |
interval |
Character string. Observation interval: '"minutely"', '"hourly"' or '"daily"'. |
station_id |
Character string. Météo-France station identifier. |
from_date |
Character string. First observation date in 'YYYY-MM-DD' format. |
to_date |
Character string. Last observation date in 'YYYY-MM-DD' format. |
The function uses the OAuth2 client-credentials flow. It submits one API command per calendar year, waits five seconds before attempting to retrieve each result, and removes columns containing only missing values. API credentials must not be hard-coded in scripts or package source files.
A data frame containing the observations returned by the API for the requested station and period.
[load_METEO()], [select_METEO_stations()]
## Not run: observations <- fct.read.meteofrance( interval = "daily", station_id = "34237001", from_date = "2020-01-01", to_date = "2020-12-31" ) ## End(Not run)## Not run: observations <- fct.read.meteofrance( interval = "daily", station_id = "34237001", from_date = "2020-01-01", to_date = "2020-12-31" ) ## End(Not run)
Fit a linear, exponential or power relationship between a predictor and response, and return the fitted coefficients.
fit_model(x, y, type = "exp")fit_model(x, y, type = "exp")
x |
Numeric vector of predictor values. |
y |
Numeric vector of response values. For 'type = "power"' or 'type = "exp"', values must be positive because the response is log-transformed. |
type |
Character string selecting the model: '"lin"' for a linear model, '"exp"' for an exponential model, or '"power"' for a power law. |
Numeric vector of length two containing coefficients 'a' and 'b'. The corresponding model forms are 'y = a + b*x' ('lin'), 'y = a*exp(b*x)' ('exp') and 'y = a*x^b' ('power').
fit_model(c(1, 2, 3, 4), c(2, 4, 6, 8), type = "lin")fit_model(c(1, 2, 3, 4), c(2, 4, 6, 8), type = "lin")
Recursively list the files contained in a directory and sum their sizes.
folder_size(path)folder_size(path)
path |
Character string. Path to an existing directory. |
File sizes that cannot be read are ignored through 'na.rm = TRUE'.
Numeric scalar giving the total size of the listed files in bytes.
[bytes_to_human()]
## Not run: folder_size("OUT") bytes_to_human(folder_size("OUT")) ## End(Not run)## Not run: folder_size("OUT") bytes_to_human(folder_size("OUT")) ## End(Not run)
Download duration and size estimation models
get_duration_size_models()get_duration_size_models()
A named list containing Duration and Size model functions.
Available data sources
get_sources()get_sources()
A named list describing the supported data sources.
Retrieves soil available water capacity (AWC) spatial data from BDGSF and Info&Sols sources.
load_AWC(vWIN, path_export)load_AWC(vWIN, path_export)
vWIN |
An 'sf' or 'sfc' spatial object defining the area of interest. |
path_export |
Character string. Root directory for exported data. |
The function writes AWC files to disk.
Retrieve selected topographic, hydrographic, vegetation, building and administrative layers from the IGN BD TOPO WFS service.
load_BDTOPO(vWIN, path_export)load_BDTOPO(vWIN, path_export)
vWIN |
An 'sf' or 'sfc' spatial object. |
path_export |
Character string. Root output directory. |
The function writes available vector layers to GeoPackage files.
Retrieve French CORINE Land Cover layers.
load_CLC(vWIN, path_export)load_CLC(vWIN, path_export)
vWIN |
An 'sf' or 'sfc' spatial object. |
path_export |
Character string. Root output directory. |
The function writes available CLC vector layers.
Retrieves soil depth data for a given study area. The function supports multiple data sources, with "Base de Données Géographique des Sols de France" (BDGSF) as the default source.
load_DEPTH(vWIN, path_export)load_DEPTH(vWIN, path_export)
vWIN |
An 'sf' or 'sfc' spatial object defining the area of interest. |
path_export |
Character string. Root directory for exported data. |
The function writes DEPTH files to disk.
Retrieves daily SIM2 data via the G-EAU API.
load_ISBA(vWIN, path_export, from_date = "2023-01-01", to_date = "2023-01-31")load_ISBA(vWIN, path_export, from_date = "2023-01-01", to_date = "2023-01-31")
vWIN |
An 'sf' or 'sfc' spatial object defining the area of interest. |
path_export |
Character string. Root directory for exported data. |
from_date |
Character string. Start date (YYYY-MM-DD). |
to_date |
Character string. End date (YYYY-MM-DD). |
The function writes an ISBA csv file to disk.
Retrieve the LiDAR HD tile index intersecting a study area and download the corresponding classified point-cloud files.
load_LIDAR(vWIN, path_export)load_LIDAR(vWIN, path_export)
vWIN |
An 'sf' or 'sfc' spatial object defining the area of interest. |
path_export |
Character string. Root output directory. |
The function writes tile metadata and downloaded point-cloud files.
Select meteorological stations relevant to a study area and observation period, then download observations for each selected station.
load_METEO( vWIN, path_export, interval = "daily", from_date = "2020-01-01", to_date = as.character(Sys.Date()) )load_METEO( vWIN, path_export, interval = "daily", from_date = "2020-01-01", to_date = as.character(Sys.Date()) )
vWIN |
An 'sf' or 'sfc' spatial object defining the study area. |
path_export |
Character string. Root directory in which the 'METEO' folder, station metadata and observation CSV files are written. |
interval |
Character string. Observation interval: '"minutely"', '"hourly"' or '"daily"'. |
from_date |
Character string. First observation date in 'YYYY-MM-DD' format. |
to_date |
Character string. Last observation date in 'YYYY-MM-DD' format. Defaults to the current date. |
API credentials are read from the 'METEOFRANCE_AUTH_BASE64' environment variable. Station selection is performed by [select_METEO_stations()], and observations are retrieved with [fct.read.meteofrance()].
The function writes station metadata (GeoPackage and CSV) and one observation CSV file per selected station. It is primarily used for its side effects.
[select_METEO_stations()], [fct.read.meteofrance()]
## Not run: area <- sf::st_read("study_area.gpkg") load_METEO(area, "OUT", interval = "daily", from_date = "2020-01-01") ## End(Not run)## Not run: area <- sf::st_read("study_area.gpkg") load_METEO(area, "OUT", interval = "daily", from_date = "2020-01-01") ## End(Not run)
Retrieve elevation rasters over a study area from IGN WMS services. The function downloads the RGE ALTI high-resolution elevation model and attempts to retrieve LiDAR HD elevation products (MNH, MNS and MNT).
load_MNE(vWIN, path_export)load_MNE(vWIN, path_export)
vWIN |
An 'sf' or 'sfc' spatial object defining the area of interest. |
path_export |
Character string. Root directory in which the 'MNE' output folder is created. |
The function writes GeoTIFF elevation rasters to disk.
Retrieve OCS GE WMTS raster layers.
load_OCS(vWIN, path_export)load_OCS(vWIN, path_export)
vWIN |
An 'sf' or 'sfc' spatial object. |
path_export |
Character string. Root output directory. |
The function writes available JPEG2000 rasters.
Retrieve current RGB and infrared-colour orthophotos.
load_ORTHO(vWIN, path_export)load_ORTHO(vWIN, path_export)
vWIN |
An 'sf' or 'sfc' spatial object. |
path_export |
Character string. Root output directory. |
The function writes JPEG2000 orthophoto rasters to disk.
Query OpenStreetMap for line features tagged as 'highway=track' or 'highway=path' within the bounding box of a study area.
load_OSM(vWIN, path_export)load_OSM(vWIN, path_export)
vWIN |
An 'sf' or 'sfc' spatial object defining the area of interest. |
path_export |
Character string. Root output directory. |
The function writes an EPSG:2154 GeoPackage.
Retrieve the Plan Corps de Rue Simplifié (PCRS) WMS layer.
load_PCRS(vWIN, path_export)load_PCRS(vWIN, path_export)
vWIN |
An 'sf' or 'sfc' spatial object. |
path_export |
Character string. Root output directory. |
The function writes JPEG2000 tiles to disk.
Retrieve current and historical French agricultural parcel data.
load_RPG(vWIN, path_export)load_RPG(vWIN, path_export)
vWIN |
An 'sf' or 'sfc' spatial object. |
path_export |
Character string. Root output directory. |
The function writes agricultural and cadastral vector datasets.
The ODR produces a vector geographic layer called "Supplemented RPG" that identifies and fills in missing agricultural parcels in the official RPG using data from IGN and OSO/Cesbio.
load_RPG_supplement(vWIN, path_export)load_RPG_supplement(vWIN, path_export)
vWIN |
An 'sf' or 'sfc' spatial object defining the area of interest. |
path_export |
Character string. Root directory for exported data. |
The function writes RPG files to disk.
Build and execute an openEO batch job to retrieve Copernicus Sentinel satellite data over a spatial and temporal extent.
load_SENTINEL( vWIN, path_export, from_date = "2016-01-01", to_date = as.character(Sys.Date()), ref = "SENTINEL1_GRD", con = NULL, poll_interval = 60, timeout = 21600, skip_existing = TRUE )load_SENTINEL( vWIN, path_export, from_date = "2016-01-01", to_date = as.character(Sys.Date()), ref = "SENTINEL1_GRD", con = NULL, poll_interval = 60, timeout = 21600, skip_existing = TRUE )
vWIN |
An 'sf' or 'sfc' spatial object defining the area of interest. Its bounding box is used as the openEO spatial extent. If available, 'vWIN$id[1]' is used as the job title. |
path_export |
Character string. Root output directory. Results are downloaded under 'SENTINEL/<ref>'. |
from_date |
Character string. First acquisition date in 'YYYY-MM-DD' format. |
to_date |
Character string. Last acquisition date in 'YYYY-MM-DD' format. Defaults to the current date. |
ref |
Character string. Collection identifier: '"SENTINEL1_GRD"' or '"SENTINEL2_L2A"'. |
con |
An authenticated openEO connection returned by 'openeo::login()'. Required. |
poll_interval |
Numeric. Time in seconds between job status checks. Defaults to 60 seconds. |
timeout |
Numeric. Maximum waiting time in seconds. Defaults to 21600 seconds (6 hours). |
skip_existing |
Logical. If 'TRUE', avoids downloading dates already covered by existing results. Defaults to 'TRUE'. |
Sentinel-2 requests the B01–B09, B11, B12 and SCL bands and calculates NDVI. Sentinel-1 requests VH and VV and applies the 'sar_backscatter' process with 'sigma0-ellipsoid' coefficient and noise removal.
An authenticated openEO session is required. The function uses the explicitly supplied connection throughout the entire workflow.
Jobs are monitored using their unique identifiers. The function stops if the job fails, is cancelled, or exceeds the specified timeout.
A character vector containing the paths of downloaded files, or an informative message if no download is necessary.
## Not run: con <- openeo::connect( host = "https://openeo.dataspace.copernicus.eu" ) con <- openeo::login(con) area <- sf::st_read("study_area.gpkg") load_SENTINEL( vWIN = area, path_export = "OUT", from_date = "2024-01-01", to_date = "2024-12-31", ref = "SENTINEL2_L2A", con = con ) ## End(Not run)## Not run: con <- openeo::connect( host = "https://openeo.dataspace.copernicus.eu" ) con <- openeo::login(con) area <- sf::st_read("study_area.gpkg") load_SENTINEL( vWIN = area, path_export = "OUT", from_date = "2024-01-01", to_date = "2024-12-31", ref = "SENTINEL2_L2A", con = con ) ## End(Not run)
Retrieve the INRA soil map and COSIA land-cover rasters.
load_SOL(vWIN, path_export)load_SOL(vWIN, path_export)
vWIN |
An 'sf' or 'sfc' spatial object. |
path_export |
Character string. Root output directory. |
The function writes soil and land-cover datasets.
Download the Météo-France station metadata file when it is missing or older than a specified number of days.
load_stations_metadatas( delay = 7, url = "https://www.data.gouv.fr/api/1/datasets/r/5ec141e4-3650-4365-82b1-db459efe690c", dest = file.path(tools::R_user_dir("datalift", which = "data"), "Carte_stations_meteo.csv") )load_stations_metadatas( delay = 7, url = "https://www.data.gouv.fr/api/1/datasets/r/5ec141e4-3650-4365-82b1-db459efe690c", dest = file.path(tools::R_user_dir("datalift", which = "data"), "Carte_stations_meteo.csv") )
delay |
Numeric. Maximum age of the cached file, in days, before a fresh copy is downloaded. Defaults to 7. |
url |
Character string. URL of the station metadata resource. |
dest |
Character string. Local destination path. By default, the file is stored in the user data directory for 'datalift'. |
The destination directory is created when necessary. The default cache location is obtained with 'tools::R_user_dir("datalift", which = "data")'.
Character string giving the local path to the station metadata CSV file.
## Not run: station_file <- load_stations_metadatas() utils::read.csv(station_file) ## End(Not run)## Not run: station_file <- load_stations_metadatas() utils::read.csv(station_file) ## End(Not run)
These functions retrieve soil texture information for a given spatial feature (sf) object from a specified data source.
load_TEXTURE(vWIN, path_export)load_TEXTURE(vWIN, path_export)
vWIN |
An 'sf' or 'sfc' spatial object defining the area of interest. |
path_export |
Character string. Root directory for exported data. |
The function writes TEXTURE files to disk.
Retrieve the IGN WMTS layer describing restrictions for open-category unmanned aircraft systems and model aircraft.
load_UAV(vWIN, path_export)load_UAV(vWIN, path_export)
vWIN |
An 'sf' or 'sfc' spatial object defining the area of interest. |
path_export |
Character string. Root output directory. |
The function writes a JPEG2000 raster to disk.
Run the Shiny Application
run_app( onStart = NULL, options = list(), enableBookmarking = NULL, uiPattern = "/", ... )run_app( onStart = NULL, options = list(), enableBookmarking = NULL, uiPattern = "/", ... )
onStart |
A function that will be called before the app is actually run.
This is only needed for |
options |
Named options that should be passed to the |
enableBookmarking |
Can be one of |
uiPattern |
A regular expression that will be applied to each |
... |
arguments to pass to golem_opts. See '?golem::get_golem_options' for more details. |
Express a duration in the largest applicable unit, from seconds to years.
seconds_to_human(seconds, digits = 1)seconds_to_human(seconds, digits = 1)
seconds |
Numeric scalar. Non-negative duration in seconds. |
digits |
Integer. Number of decimal places used for rounding. |
Character string containing the rounded duration and its unit ('sec', 'min', 'hours', 'days', 'week' or 'year'). A year is defined as 365.25 days.
seconds_to_human(90) seconds_to_human(86400 * 10, digits = 2)seconds_to_human(90) seconds_to_human(86400 * 10, digits = 2)
Filter the Météo-France station catalogue by department, station type, observation period and temporal resolution. Stations located within the study area are selected; if none are inside, the nearest available station is returned.
select_METEO_stations( vWIN, interval = "daily", from_date = "2016-01-01", to_date = as.character(Sys.Date()) )select_METEO_stations( vWIN, interval = "daily", from_date = "2016-01-01", to_date = as.character(Sys.Date()) )
vWIN |
An 'sf' or 'sfc' spatial object defining the study area. |
interval |
Character string. Observation interval: '"minutely"', '"hourly"' or '"daily"'. |
from_date |
Character string. First date of the requested period in 'YYYY-MM-DD' format. |
to_date |
Character string. Last date of the requested period in 'YYYY-MM-DD' format. Defaults to the current date. |
Station metadata are obtained through [load_stations_metadatas()]. The station catalogue is filtered to mainland departments up to 95, excludes station '17300848' and station type 5, and uses the 'is_<interval>' field to check temporal-resolution availability.
An 'sf' object containing the selected meteorological station or stations, with point geometries in the station catalogue CRS.
[load_METEO()], [load_stations_metadatas()]
## Not run: area <- sf::st_read("study_area.gpkg") stations <- select_METEO_stations(area, interval = "daily", from_date = "2020-01-01") ## End(Not run)## Not run: area <- sf::st_read("study_area.gpkg") stations <- select_METEO_stations(area, interval = "daily", from_date = "2020-01-01") ## End(Not run)