Skip to contents

Check Integrity of Data Files

Source: R/general_check_data.R
check_data.Rd

Validates a data file by checking its extension and attempting to load its contents. A file is considered valid if it exists, is non-empty, has a supported extension, and loads successfully with a non-null object. Supports RData, qs2, rds, and feather file types.

Usage

check_data(file = NULL, warning = TRUE, n_threads = 1L)

check_rdata(file, warning = TRUE)

check_qs(file, warning = TRUE, n_threads = 1L)

check_rds(file, warning = TRUE)

check_feather(file, warning = TRUE)

Arguments

file

Character. Path to a data file (e.g., .rdata, .qs2, .rds, .feather). Must be a single, non-empty string.

warning

Logical. If TRUE (default), warnings are issued for invalid files (e.g., non-existent, wrong extension, or loading failure).

n_threads

Integer. Number of threads for reading qs2 files. Must be a positive integer. See qs2::qs_read for more details.

Value

Logical: TRUE if the file is valid and loads successfully; FALSE otherwise, with a warning if warning = TRUE.

Details

The check_data() function determines the file type based on its extension (case-insensitive). If the extension is unrecognised, it returns FALSE. Supported file types:

  • RData: Checked with check_rdata(), read using load_as

  • qs2: Checked with check_qs(), read using qs2::qs_read

  • rds: Checked with check_rds(), read using readRDS

  • feather: Checked with check_feather(), read using arrow::read_feather

Author

Ahmed El-Gabbas

Examples


# Setup temporary directory
temp_dir <- fs::path_temp("load_multiple")
fs::dir_create(temp_dir)

# |||||||||||||||||||||||||||||||||||||||
# Validate RData files
# |||||||||||||||||||||||||||||||||||||||

# valid RData file
data <- data.frame(x = 1:5)
rdata_file <- fs::path(temp_dir, "valid.Rdata")
save(data, file = rdata_file)

# Invalid RData file (corrupted)
bad_rdata <- fs::path(temp_dir, "invalid.Rdata")
writeLines("not an RData file", bad_rdata)

check_data(rdata_file)                               # TRUE
#> [1] TRUE
check_rdata(rdata_file)                              # TRUE
#> [1] TRUE

check_data(bad_rdata)                                # FALSE, with warning
#> Warning: Not a valid RData file: /tmp/RtmphTiYIO/load_multiple/invalid.Rdata
#> [1] FALSE
check_rdata(bad_rdata)                               # FALSE, with warning
#> Warning: Not a valid RData file: /tmp/RtmphTiYIO/load_multiple/invalid.Rdata
#> [1] FALSE

check_data(bad_rdata, warning = FALSE)               # FALSE, no warning
#> [1] FALSE
check_rdata(bad_rdata, warning = FALSE)              # FALSE, no warning
#> [1] FALSE

# |||||||||||||||||||||||||||||||||||||||
# Validate qs2 files
# |||||||||||||||||||||||||||||||||||||||

# Valid qs2 file
qs_file <- fs::path(temp_dir, "valid.qs2")
qs2::qs_save(data, qs_file, nthreads = 1)

# Invalid qs2 file (corrupted)
bad_qs <- fs::path(temp_dir, "invalid.qs2")
writeLines("not a qs2 file", bad_qs)

check_data(qs_file, n_threads = 1L)                  # TRUE
#> [1] TRUE
check_qs(qs_file, n_threads = 1L)                    # TRUE
#> [1] TRUE

check_data(bad_qs, n_threads = 1L)                   # FALSE, with warning
#> Warning: Not a valid qs2 file: /tmp/RtmphTiYIO/load_multiple/invalid.qs2
#> [1] FALSE
check_qs(bad_qs, n_threads = 1L)                     # FALSE, with warning
#> Warning: Not a valid qs2 file: /tmp/RtmphTiYIO/load_multiple/invalid.qs2
#> [1] FALSE

# |||||||||||||||||||||||||||||||||||||||
# Validate rds files
# |||||||||||||||||||||||||||||||||||||||

# Valid rds file
rds_file <- fs::path(temp_dir, "valid.rds")
saveRDS(data, rds_file)

# Invalid rds file (corrupted)
bad_rds <- fs::path(temp_dir, "invalid.rds")
writeLines("not an rds file", bad_rds)

check_data(rds_file)                                 # TRUE
#> [1] TRUE
check_rds(rds_file)                                  # TRUE
#> [1] TRUE

check_data(bad_rds)                                  # FALSE, with warning
#> Warning: Not a valid rds file: /tmp/RtmphTiYIO/load_multiple/invalid.rds
#> [1] FALSE
check_rds(bad_rds)                                   # FALSE, with warning
#> Warning: Not a valid rds file: /tmp/RtmphTiYIO/load_multiple/invalid.rds
#> [1] FALSE

# |||||||||||||||||||||||||||||||||||||||
# Validate feather files
# |||||||||||||||||||||||||||||||||||||||

# Valid feather file
feather_file <- fs::path(temp_dir, "valid.feather")
arrow::write_feather(data, feather_file)

# Invalid feather file (corrupted)
bad_feather <- fs::path(temp_dir, "invalid.feather")
writeLines("not a feather file", bad_feather)

check_data(feather_file)                             # TRUE
#> [1] TRUE
check_feather(feather_file)                          # TRUE
#> [1] TRUE

check_data(bad_feather)                              # FALSE, with warning
#> Warning: Not a valid feather file: /tmp/RtmphTiYIO/load_multiple/invalid.feather
#> [1] FALSE
check_feather(bad_feather)                           # FALSE, with warning
#> Warning: Not a valid feather file: /tmp/RtmphTiYIO/load_multiple/invalid.feather
#> [1] FALSE

# |||||||||||||||||||||||||||||||||||||||
# Non-existent file
# |||||||||||||||||||||||||||||||||||||||

check_data("nonexistent.rds")                        # FALSE, with warning
#> Warning: File does not exist: `/home/runner/work/ecokit/ecokit/docs/reference/nonexistent.rds`
#> [1] FALSE

# Clean up
fs::file_delete(
  c(rdata_file, bad_rdata, qs_file, bad_qs, rds_file, bad_rds,
  feather_file, bad_feather))
fs::dir_delete(temp_dir)