Skip to content

Repository for contributing, validating and processing data in the doppa environment

License

Notifications You must be signed in to change notification settings

kartAI/doppa-data

Repository files navigation

doppa: A Framework for Comparing Traditional & Cloud-Native Geospatial Queries

Build and Push Query- and Orchestration-containers to Azure Container Registry Run Benchmarks

Setup

Create a virtual environment and install the dependencies in the requirements-file.

Add the following .env file in the root

AZURE_BLOB_STORAGE_CONNECTION_STRING=<azure-blob-storage-connection-string>
AZURE_BLOB_STORAGE_BENCHMARK_CONTAINER=dev-benchmarks
AZURE_BLOB_STORAGE_METADATA_CONTAINER=dev-metadata
ACR_LOGIN_SERVER=<azure-container-registry-login-server>
ACR_USERNAME=<azure-container-registry-username>
ACR_PASSWORD=<azure-container-registry-password>
POSTGRES_USERNAME=<postgres-username>
POSTGRES_PASSWORD=<postgres-password>

Note

Ensure that the needed Azure Resources have been configured

To run the entire script simply run python main.py or python -m main and to run a single benchmark run python benchmark_runner.py --script-id <script-id> --benchmark-run <int >= 1> --run-id <run-id>. See the table below for more information about --script-id and --run-id.

Flag Format / Pattern Meaning
--script-id <query-type>-<service> Identifies which query is being executed. <query-type> examples: db-scan, bbox-filtering. <service> examples: blob-storage, postgis.
--benchmark-run int Identifier that tells which iteration of the benchmarking is currently running. This is to run the benchmarks on multiple container instances.
--run-id <current-date>-<random-id> Identifies a benchmark run. Shared across all queries in a single orchestrated run. Date format: yyyy-mm-dd; random ID: 6-character uppercase alphanumeric.

About

Repository for contributing, validating and processing data in the doppa environment

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors