geolife example

[mdsumner]

library(trip)

dp <- "geolife/Geolife Trajectories 1.3/Data"
fs <- list.files(dp, pattern = "plt$", full.names = TRUE, recursive = TRUE)
readLines(fs[1], n = 10)
library(readr)
library(dplyr)
read_csv(x,  col_names = c("lat", "lon", "zero", "altitude", "date1", "date2", "time"))

system.time({
d <- bind_rows(lapply(fs, read_csv, 
                      col_names = c("lat", "lon", "zero", "altitude", "date1", "date2", "time"), 
                      col_types = cols(lat = "d", lon = "d", zero = "i", altitude = "d", date1 = "d", date2 = "c", time = "c"), 
                      skip = 6))
})
# user  system elapsed 
#492.344 110.476 644.266 

d$utc <- as.POSIXct(strptime(paste(d$date2, d$time), "%Y-%m-%d %H:%M:%S"),  tz = "UTC")
saveRDS(d, file = "geolife.rds", compress = "xz")
d <- readRDS("geolife.rds")
idx  <- seq(1, nrow(d), length = 1e6)
d <- d[idx, ]

library(palr)
library(viridis)
tpal <- mk_timePal(d$utc, col = viridis::viridis(64))

library(rgdal)
xy <- project(cbind(d$lon, d$lat), "+proj=laea +lon_0=140")
library(maptools)
data(wrld_simpl)
m <- spTransform(wrld_simpl, "+proj=laea +lon_0=140")

plot(xy, pch = ".", col = tpal(d$utc))
plot(m, add = FALSE)
points(xy, pch = ".", col = tpal(d$utc))

[mdsumner]

R handles this in memory but everything takes too long, dplyr summaries and reprojecting are fine but load and plot and so forth are too slow. (Plus, I forgot to .id the separate trips in bind_rows with the file name).

Second round should:

• upload all to a sqlite db with decomposed date time (drop the zero, and the numeric date)
• incrementally add to the "trip" table as well with link (main table starts as v0 but will be bXv).
• find distinct long lat and create v table (check if altitude changes distinctness?)
• keep lon, lat, altitude, year, month, day, hour, min, sec, tripID columns in bXv
• Use links on trip to get min max coords
• plot "gantt" charts of lon-lat-date agst tripID
• build time spent maps for various windows