saferoad.pipeline module

class saferoad.pipeline.Pipeline

Bases: object

class Processing

Bases: object

Processing class for processing road data.

This class provides methods to load files into a database, dissolve features, split lines into segments, generate patches, and save tables as GeoJSON files. It assumes that the database connection is already established and that the necessary SQL functions for spatial operations are available. The methods in this class generate SQL queries that can be executed against a spatial database, such as DuckDB.

The class does not handle database connections or query execution directly; it only generates the SQL queries as strings. The methods are static and can be called without instantiating the class. The class is designed to be used in a pipeline for processing road data, where each method can be chained together to perform a series of operations on the road data.

The methods assume that the input data is already in a suitable format for processing, such as a GIS vector file for loading into a database or a table containing road geometries for dissolving and segmenting. The class is intended to be used in conjunction with a spatial database that supports SQL queries for spatial operations, such as DuckDB with the Spatial extension or PostGIS. The methods are designed to be flexible and can be adapted to different use cases by modifying the parameters passed to them.

static build_geometry(latitude: str, longitude: str, table_name: str) → str

Build a geometry from latitude and longitude coordinates.

This method generates an SQL query to build a geometry column in a table from latitude and longitude coordinates. It assumes that the input table contains columns for latitude and longitude.

Parameters:

• latitude (str) – The name of the column containing latitude values.

• longitude (str) – The name of the column containing longitude values.

• table_name (str) – The name of the table to add the geometry column to.

Returns:

SQL query string to build the geometry column.

Return type:

str

static calc_avg_cum_disp() → str

Calculate average cumulative displacement per patch.

This method generates an SQL query to calculate the average cumulative displacement for each patch by averaging the displacement values of points within each patch. It assumes that the point table contains a geometry column, a displacement column, and that the patch table contains a geometry column and a unique identifier for each patch.

Returns:

SQL query string to calculate average cumulative displacement per patch.

Return type:

str

static calc_avg_disp_ts_gcp(date_fields: list[str])

Calculate average displacement time series for patches (Average Local Displacement (ALD)).

This method generates an SQL query to calculate the average displacement time series for each patch by averaging the displacement values of points within each patch for the specified date fields. It assumes that the point table contains a geometry column, displacement columns for each date field, and that the patch table contains a geometry column and a unique identifier for each patch.

Parameters:

date_fields (list[str]) – List of column names representing displacement values at different dates.

Returns:

SQL query string to calculate average displacement time series per patch.

Return type:

str

static calc_avg_lcp_velocity(table_name: str = 'pspoint') → str

Calculate average LCP velocity per patch.

This method generates an SQL query to calculate the average velocity of local control points (LCPs) for each patch by averaging the average velocities of points within each patch based on their LCP velocities.

Parameters:

table_name (str) – The name of the table containing the points, default is “pspoint”.

Returns:

SQL query string to calculate average LCP velocity per patch.

Return type:

str

static calc_avg_std_disp_ts_lcp(date_fields: list[str], table_name: str = 'pspoint') → str

Calculate average and standard deviation of displacement time series based on LCPs.

This method generates an SQL query to calculate the average and standard deviation of displacement time series for each patch based on the displacement values of points within each patch relative to the local control point (LCP). It assumes that the point table contains columns for displacement values at different dates and that the patch table contains a geometry column and a unique identifier for each patch.

Parameters:

• date_fields (list[str]) – List of column names representing displacement values at different dates.

• table_name (str) – The name of the table containing the points, default is “pspoint”.

Returns:

SQL query string to calculate average and standard deviation of displacement time series based on LCPs.

Return type:

str

static calc_avg_velocity(time_vector: list[float], field_names: list[str], table_name: str = 'pspoint') → str

Calculate average velocity using linear regression.

This method generates an SQL query to calculate the average velocity for each point in a table using linear regression on displacement values over time. It assumes that the input table contains columns for displacement values at different dates and that the time vector is provided in days.

Parameters:

• time_vector (list[float]) – List of time differences in days from the first date.

• field_names (list[str]) – List of column names representing displacement values at different dates.

• table_name (str) – The name of the table containing the points, default is “pspoint”.

Returns:

SQL query string to calculate average velocity.

Return type:

str

static calc_avg_velocity_gcp(table_name: str = 'pspoint') → str

Calculate average velocity based on ground control points (GCPs) for patches.

This method generates an SQL query to calculate the average velocity for each patch based on the average velocities of points within each patch.

Parameters:

table_name (str) – The name of the table containing the points, default is “pspoint”.

Returns:

SQL query string to calculate average velocity based on GCPs for patches.

Return type:

str

static calc_avg_velocity_lcp(table_name: str = 'pspoint') → str

Calculate average velocity based on local control points (LCPs) for points.

This method generates an SQL query to calculate the average velocity for each point based on the average velocity of the local control point (LCP) associated with the patch to which the point belongs. It assumes that the point table contains a geometry column, an average velocity column based on ground control points (GCPs), and that the patch table contains a geometry column, a unique identifier for each patch, and the average velocity of the LCP.

Parameters:

table_name (str) – The name of the table containing the points, default is “pspoint”.

Returns:

SQL query string to calculate average velocity based on LCPs for points.

Return type:

str

static calc_cum_disp(dates: list[str], table_name: str = 'pspoint') → str

Calculate cumulative displacement.

This method generates an SQL query to calculate the cumulative displacement for each point in a table based on the provided date fields. It assumes that the input table contains columns for displacement values at different dates.

Parameters:

• dates (list[str]) – List of column names representing dates.

• table_name (str) – The name of the table containing the points, default is “pspoint”.

Returns:

SQL query string to calculate cumulative displacement.

Return type:

str

static calc_lcp_velocity() → str

Calculate LCP velocity for patches.

This method generates an SQL query to calculate the velocity of the linear control point (LCP) for each patch by retrieving the average velocity of the point identified as the LCP.

Returns:

SQL query string to calculate LCP velocity for patches.

Return type:

str

static calc_med_std_disp_ts_lcp(date_fields: list[str], table_name: str = 'pspoint') → str

Calculate average and standard deviation of displacement time series based on LCPs.

This method generates an SQL query to calculate the average and standard deviation of displacement time series for each patch based on the displacement values of points within each patch relative to the local control point (LCP). It assumes that the point table contains columns for displacement values at different dates and that the patch table contains a geometry column and a unique identifier for each patch.

Parameters:

• date_fields (list[str]) – List of column names representing displacement values at different dates.

• table_name (str) – The name of the table containing the points, default is “pspoint”.

Returns:

SQL query string to calculate average and standard deviation of displacement time series based on LCPs.

Return type:

str

static calc_outliers(scaling_factor: float, table_name: str = 'pspoint') → str

Identify outlier points based on absolute average velocity.

This method generates an SQL query to identify outlier points based on their average velocity relative to local control points (LCPs). Points with an absolute average LCP velocity exceeding a specified threshold (5 mm/year) are marked as outliers by assigning their unique identifier to a new column in the patches table. The unit parameter allows for specifying the measurement unit (meters, centimeters, or millimeters) to appropriately scale the threshold.

Parameters:

• scaling_factor (float) – The factor to scale the velocity threshold based on the unit.

• table_name (str) – The name of the table containing the points, default is “pspoint”.

Returns:

SQL query string to identify outlier points.

Return type:

str

static calc_outliers_avg_lcd(table_name: str = 'pspoint', date_vector=list[int], date_names=list[str]) → str

Calculate outliers based on average local control displacement (LCD).

This method generates an SQL query to identify outlier points based on their displacement time series relative to the average local control displacement (LCD) of their associated patches. Points with displacement values exceeding the average LCD by more than standard deviations at any time point are marked as outliers by inserting their unique identifier and patch ID into a new outliers table.

Parameters:

• table_name (str) – The name of the table containing the points, default is “pspoint”.

• date_vector (list[int]) – List of time differences in days from the first date.

• date_names (list[str]) – List of column names representing displacement values at different dates.

Returns:

SQL query string to identify outlier points based on average LCD.

Return type:

str

static calc_outliers_med_lcd(table_name: str = 'pspoint', date_vector=list[int], date_names=list[str]) → str

Calculate outliers based on median local control displacement (LCD).

This method generates an SQL query to identify outlier points based on their displacement time series relative to the median local control displacement (LCD) of their associated patches. Points with displacement values exceeding the median LCD by more than standard deviations at any time point are marked as outliers by inserting their unique identifier and patch ID into a new outliers table.

Parameters:

• table_name (str) – The name of the table containing the points, default is “pspoint”.

• date_vector (list[int]) – List of time differences in days from the first date.

• date_names (list[str]) – List of column names representing displacement values at different dates.

Returns:

SQL query string to identify outlier points based on median LCD.

Return type:

str

static calc_point_density(point_name: str, patch_name: str = 'patches') → str

Calculate point density per patch.

This method generates an SQL query to calculate the point density per patch by counting the number of points within each patch and dividing by the area of the patch in square kilometers. It assumes that the point table contains a geometry column and that the patch table contains a geometry column and a unique identifier for each patch.

Parameters:

• point_name (str) – The name of the table containing the points.

• patch_name (str) – The name of the table containing the patches, default is “patches”.

Returns:

SQL query string to calculate point density per patch.

Return type:

str

static calc_std_disp_ts_gcp(date_fields: list[str])

Calculate standard deviation of displacement time series for patches based on global control point (GCP).

This method generates an SQL query to calculate the standard deviation of displacement time series for each patch by computing the standard deviation of displacement values of points within each patch for the specified date fields.

Parameters:

date_fields (list[str]) – List of column names representing displacement values at different dates.

Returns:

SQL query string to calculate standard deviation of displacement time series per patch.

Return type:

str

static dissolve_features(attribute: str, table_name: str) → str

Dissolve features in a table based on a specified attribute.

This method generates an SQL query to dissolve features in a table based on the provided attribute. It assumes the table is already loaded into the database.

Parameters:

• attribute (str) – The attribute to dissolve features by.

• table_name (str) – The name of the table containing the features, default is “road”.

Returns:

SQL query string to dissolve features.

Return type:

str

static generate_patches(width: float, length: float, table_name: str = 'segments') → str

Generate patches from segments.

This method generates patches from segments by creating a rectangular envelope around each segment, rotated to align with the segment’s direction. The patches are created by translating and rotating the envelope to the start point of the segment, ensuring that each patch is oriented correctly along the segment’s direction.

Parameters:

• width (float) – The width of the patches in meters.

• length (float) – The length of the patches in meters.

• table_name (str) – The name of the table containing the segments, default is “segments”.

Returns:

SQL query string to create or replace the patches table.

Return type:

str

static get_column_names(table_name: str) → str

Get column names from a table.

This method generates an SQL query to retrieve the column names from a specified table.

Parameters:

table_name (str) – The name of the table to retrieve column names from.

Returns:

SQL query string to get the column names.

Return type:

str

static lcp_ts_patch(date_fields: list[str], table_name: str = 'pspoint')

Get Local Control Point’s displacement time series for patches.

This method generates an SQL query to retrieve the displacement time series of the local control point (LCP) for each patch based on specified date fields.

Parameters:

• date_fields (list[str]) – List of column names representing displacement values at different dates.

• table_name (str) – The name of the table containing the points, default is “pspoint”.

Returns:

SQL query string to get LCP displacement time series for patches.

Return type:

str

static load_file(file_path: str, table_name: str) → str

Load a file into a database table.

Method validates the provided file path and determines if existing and determines the appropriate loading method based on the file type. It generates an SQL query to create or replace a table in the database with the contents of the file. The method supports CSV files and other GIS vector files, using different loading methods for each type.

Parameters:

• file_path (str) – The path to the file to be loaded.

• table_name (str) – The name of the table to create or replace in the database.

Raises:

AssertionError – If the file does not exist.

Returns:

SQL query string to create or replace the table with the file data.

Return type:

str

static outlier_rel_ts_patch()

Calculate outlier displacement time series relative to LCP for patches.

This method generates an SQL query to calculate the displacement time series of outlier points relative to the displacement time series of their associated local control points (LCPs). It assumes that the outliers table contains a geometry column, a displacement time series column, and a patch ID column, and that the patches table contains a geometry column, a unique identifier for each patch, and a displacement time series column for the LCP.

Returns:

SQL query string to calculate outlier displacement time series relative to LCP for patches.

Return type:

str

outlier_ts_patch(table_name: str = 'pspoint')

Get outlier displacement time series for patches.

This method generates an SQL query to retrieve the displacement time series of outlier points based on specified date fields.

Parameters:

• date_fields (list[str]) – List of column names representing displacement values at different dates.

• table_name (str) – The name of the table containing the points, default is “pspoint”.

Returns:

SQL query string to get outlier displacement time series for patches.

Return type:

str

static relate_point_patches(point_name: str, patch_name: str = 'patches') → str

Relate points to patches by assigning patch IDs to points.

This method generates an SQL query to relate points in one table to patches in another table by assigning patch IDs to points based on their spatial relationship. It assumes that the point table contains a geometry column and that the patch table contains a geometry column and a unique identifier for each patch.

Parameters:

• point_name (str) – The name of the table containing the points.

• patch_name (str) – The name of the table containing the patches, default is “patches”.

Returns:

SQL query string to relate points to patches.

Return type:

str

static select_lcps(table_name: str = 'pspoint') → str

Select linear control points (LCPs) for patches.

This method generates an SQL query to select local control points (LCPs) for each patch by identifying the point with the minimum absolute average velocity within each patch.

Parameters:

table_name (str) – The name of the table containing the points, default is “pspoint”.

Returns:

SQL query string to select LCPs for patches.

Return type:

str

static spatial_tranformation(source_epsg: str, target_epsg: str, table_name: str) → str

Transform geometries from one EPSG code to another.

This method generates an SQL query to transform geometries in a table from one EPSG code to another. It assumes that the input table contains geometries in the source EPSG code and that the target EPSG code is valid.

Parameters:

• source_epsg (str) – The source EPSG code of the geometries.

• target_epsg (str) – The target EPSG code to transform the geometries to.

• table_name (str) – The name of the table containing the geometries, default is “patches”.

Returns:

SQL query string to transform the geometries.

Return type:

str

static split_to_segments(segment_length: float, table_name: str = 'dissolved') → str

Split lines into segments of a specified length.

This method generates an SQL query to split lines in a table into segments of a specified length. It transforms the geometries to a computational coordinate reference system (CRS) and calculates the lengths in meters. The segments are created by generating windows of fixed length along the lines. The resulting segments are stored in a new table named “segments”. The method assumes that the input table is already dissolved and contains geometries in a suitable format.

The segments are created by cutting the lines into fixed-length segments and filtering out any empty geometries. This method is useful for preparing road segments for further analysis or processing. It is particularly useful in road network analysis, where roads need to be divided into manageable segments for tasks such as traffic analysis, road condition assessment, or infrastructure planning.

Parameters:

• segment_length (float) – The length of each segment in meters.

• table_name (str) – The name of the table containing the lines, default is “dissolved”.

Returns:

SQL query string to create or replace the segments table.

Return type:

str

class Visualisation

Bases: object

SQL queries for visualisation of results.

2D visualisation is done in Web Mercator (EPSG:3857).

static get_avg_cum_disp_graph(patch_id: float, scaling_factor: float) → str

Get average cumulative displacement time series.

This method generates an SQL query to retrieve the average cumulative displacement time series for a specific patch, scaling the displacement values by a specified factor.

Parameters:

• patch_id (float) – The unique identifier of the patch.

• scaling_factor (float) – The factor to scale the displacement values.

Returns:

SQL query string to get the average cumulative displacement time series for the specified patch.

static get_avg_disp_lcp_graph(patch_id: float, scaling_factor: float) → str

Get average displacement time series based on local control points (LCPs).

This method generates an SQL query to retrieve the average displacement time series based on local control points (LCPs) for a specific patch, scaling the displacement values by a specified factor.

Parameters:

• patch_id (float) – The unique identifier of the patch.

• scaling_factor (float) – The factor to scale the displacement values.

Returns:

SQL query string to get the average displacement time series based on LCPs for the specified patch.

static get_center(point_uid: float, source_crs: str) → str

Get the center point of the outlier within a specific patch.

This method generates an SQL query to retrieve the center point of the outlier within a specific patch, transforming the geometry from a source CRS to a target projection defined in the Visualisation class.

Parameters:

• point_uid (float) – The unique identifier of the outlier point.

• source_crs (str) – The source coordinate reference system of the geometries.

Returns:

SQL query string to get the center point of the outlier within the specified patch.

static get_cum_disp_graph(outlier_id: float, scaling_factor: float) → str

Get outlier displacement time series.

This method generates an SQL query to retrieve the outlier displacement time series for a specific patch, scaling the displacement values by a specified factor.

Parameters:

• outlier_id (float) – The unique identifier of the patch.

• scaling_factor (float) – The factor to scale the displacement values.

Returns:

SQL query string to get the outlier displacement time series for the specified patch.

static get_cum_disp_map(source_crs: str, scaling_factor: float) → str

Get cumulative displacement per patch.

This method generates an SQL query to retrieve the cumulative displacement for each patch, transforming the geometries from a source CRS to a target projection defined in the Visualisation class.

Parameters:

• source_crs (str) – The source coordinate reference system of the geometries.

• unit (str) – The measurement unit for displacement (“m”, “cm”, or “mm”).

Returns:

SQL query string to get the cumulative displacement per patch.

Return type:

str

static get_lcp_disp_graph(patch_id: float, scaling_factor: float) → str

Get local control point (LCP) displacement time series.

This method generates an SQL query to retrieve the local control point (LCP) displacement time series for a specific patch, scaling the displacement values by a specified factor.

Parameters:

• patch_id (float) – The unique identifier of the patch.

• scaling_factor (float) – The factor to scale the displacement values.

Returns:

SQL query string to get the LCP displacement time series for the specified patch.

static get_lcp_point(patch_id: float, source_crs: str) → str

Get the local control point (LCP) within a specific patch.

This method generates an SQL query to retrieve the local control point (LCP) within a specific patch, transforming the geometry from a source CRS to a target projection defined in the Visualisation class.

Parameters:

• patch_id (float) – The unique identifier of the patch.

• source_crs (str) – The source coordinate reference system of the geometries.

Returns:

SQL query string to get the LCP within the specified patch.

static get_outlier_points(patch_id: float, source_crs: str) → str

Get the outlier points within a specific patch.

This method generates an SQL query to retrieve the outlier point within a specific patch, transforming the geometry from a source CRS to a target projection defined in the Visualisation class.

Parameters:

• patch_id (float) – The unique identifier of the patch.

• source_crs (str) – The source coordinate reference system of the geometries.

Returns:

SQL query string to get the outlier point within the specified patch.

get_outlier_rel_ts_graph(scaling_factor: float) → str

Get outlier relative displacement time series.

This method generates an SQL query to retrieve the outlier relative displacement time series for a specific patch, scaling the displacement values by a specified factor.

Parameters:

• outlier_id (float) – The unique identifier of the patch.

• scaling_factor (float) – The factor to scale the displacement values.

Returns:

SQL query string to get the outlier relative displacement time series for the specified patch.

static get_outliers_map(source_crs: str, scaling_factor: float) → str

Get outlier points based on absolute average LCP velocity.

This method generates an SQL query to retrieve outlier points based on their average velocity relative to local control points (LCPs). Points with an absolute average LCP velocity exceeding a specified threshold (5 mm/year) are marked as outliers by assigning their unique identifier to a new column in the patches table. The unit parameter allows for specifying the measurement unit (meters, centimeters, or millimeters) to appropriately scale the threshold.

Parameters:

• source_crs (str) – The source coordinate reference system of the geometries.

• scaling_factor (float) – The factor to scale the velocity threshold based on the unit.

Returns:

SQL query string to identify outlier points.

Return type:

str

static get_patch_center(patch_id: float, source_crs: str) → str

Get the center point of a specific patch.

This method generates an SQL query to retrieve the center point of a specific patch, transforming the geometry from a source CRS to a target projection defined in the Visualisation class.

Parameters:

• patch_id (float) – The unique identifier of the patch.

• source_crs (str) – The source coordinate reference system of the geometries.

Returns:

SQL query string to get the center point of the specified patch.

static get_ps_density(source_crs: str) → str

Get point density per patch.

This method generates an SQL query to retrieve the point density for each patch, transforming the geometries from a source CRS to a target projection defined in the Visualisation class.

Parameters:

source_crs (str) – The source coordinate reference system of the geometries.

Returns:

SQL query string to get the point density per patch.

Return type:

str

static get_ps_points(patch_id: float, source_crs: str, table_name: str) → str

Get points within a specific patch.

This method generates an SQL query to retrieve points within a specific patch, transforming the geometries from a source CRS to a target projection defined in the Visualisation class.

Parameters:

• patch_id (float) – The unique identifier of the patch.

• source_crs (str) – The source coordinate reference system of the geometries.

• table_name (str) – The name of the table containing the points.

Returns:

SQL query string to get the points within the specified patch.

Return type:

str

static get_ps_vel_gcp(source_crs: str, scaling_factor: float) → str

Get average velocity based on ground control points (GCPs). This method generates an SQL query to retrieve the average velocity for each point based on ground control points (GCPs), transforming the geometries from a source CRS to a target projection defined in the Visualisation class.

Parameters:

• source_crs (str) – The source coordinate reference system of the geometries.

• unit (str) – The measurement unit for velocity (“m”, “cm”, or “mm”).

Returns:

SQL query string to get the average velocity based on GCPs.

Return type:

str

static get_ps_vel_lcp(source_crs: str, scaling_factor: float) → str

Get average velocity based on local control points (LCPs).

This method generates an SQL query to retrieve the average velocity for each point based on local control points (LCPs), transforming the geometries from a source CRS to a target projection defined in the Visualisation class.

Parameters:

• source_crs (str) – The source coordinate reference system of the geometries.

• unit (str) – The measurement unit for velocity (“m”, “cm”, or “mm”).

Returns:

SQL query string to get the average velocity based on LCPs.

Return type:

str

static get_std_cum_disp_graph(patch_id: float, scaling_factor: float) → str

Get standard deviation of cumulative displacement time series.

This method generates an SQL query to retrieve the standard deviation of cumulative displacement time series for a specific patch, scaling the displacement values by a specified factor.

Parameters:

• patch_id (float) – The unique identifier of the patch.

• scaling_factor (float) – The factor to scale the displacement values.

Returns:

SQL query string to get the standard deviation of cumulative displacement time series for the specified patch.

static get_std_disp_lcp_graph(patch_id: float, scaling_factor: float) → str

Get standard deviation of displacement time series based on local control points (LCPs).

This method generates an SQL query to retrieve the standard deviation of displacement time series based on local control points (LCPs) for a specific patch, scaling the displacement values by a specified factor.

Parameters:

• patch_id (float) – The unique identifier of the patch.

• scaling_factor (float) – The factor to scale the displacement values.

Returns:

SQL query string to get the standard deviation of displacement time series based on LCPs for the specified patch.

static get_table_bbox(table_name: str, source_crs: str) → str

Get the bounding box of a table in a specified projection.

This method generates an SQL query to retrieve the bounding box of geometries from a specified table, transforming the geometries from a source CRS to a target projection defined in the Visualisation class.

Parameters:

• table_name (str) – The name of the table to get the bounding box from.

• source_crs (str) – The source coordinate reference system of the geometries.

Returns:

SQL query string to get the bounding box of the table.

Return type:

str

static patch_uids() → str

Get all patch UIDs that have outliers.

This method generates an SQL query to retrieve all unique identifiers (UIDs) of patches that have been identified as containing outliers.

Returns:

SQL query string to get all patch UIDs with outliers.

Return type:

str

projection = 'EPSG:3857'