GUNDI-4684: Vectronic migration script (v1.0)

[vgarcia13]

Relevant Link

https://allenai.atlassian.net/browse/GUNDI-4684

---

This pull request introduces a new Django management command script, vectronic_v1_migration_script.py, to automate the migration of Vectronic v1 plugin data to the Gundi v2 integration system. The script reads plugin and destination data from JSON files, processes and deduplicates collar records, and creates or updates corresponding integrations, configurations, and routes in the database. It supports filtering by site, limiting the number of plugins migrated, and outputs a migration summary.

Key changes include:

Migration Script Implementation:

• Added a new management command vectronic_v1_migration_script.py that migrates Vectronic v1 plugins to Gundi v2 by reading from JSON files, deduplicating collar data, and creating/updating Integration, IntegrationConfiguration, Route, and RouteConfiguration records. The script handles both plugin and destination integrations, sets up default routes, and manages field mappings for data providers.

Command-line Interface and Options:

• Introduced command-line arguments to allow migration of all plugins or only those from specified sites, and to limit the number of plugins processed. Outputs progress and summary information to the console.

Error Handling and Logging:

• Integrated error handling for missing integration types, actions, or destinations, and logs warnings and errors during migration. Provides a summary of created, skipped, and errored migrations at the end of execution.

[Copilot]

Pull Request Overview

This pull request introduces a Django management command to migrate Vectronic v1 plugin data to Gundi v2 integration system. The script processes plugin and destination data from JSON files, handles collar records deduplication, and creates corresponding integrations, configurations, and routes in the database.

• Adds comprehensive migration logic for transforming v1 plugin data into v2 integration structures
• Implements command-line interface with options for site filtering and migration limits
• Includes error handling, logging, and migration summary reporting

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Copilot]

This TODO comment indicates temporary code that should be removed for production deployment. Consider making this configurable through command-line arguments or environment variables instead of hardcoding it.

Suggested change

	def handle(self, *args, **options):
	# TODO: FOR LOCAL EXECUTION ONLY! Remove if running in pod
	logging.getLogger('django.db.backends').setLevel(logging.WARNING)
	logging.getLogger('activity_log.mixins').setLevel(logging.ERROR)
	logging.getLogger('integrations.tasks').setLevel(logging.WARNING)
	parser.add_argument(
	"--set-logger-levels",
	action="store_true",
	default=False,
	help="If present, set local logger levels for development (not for production use)",
	)
	def handle(self, *args, **options):
	if options.get("set_logger_levels"):
	logging.getLogger('django.db.backends').setLevel(logging.WARNING)
	logging.getLogger('activity_log.mixins').setLevel(logging.ERROR)
	logging.getLogger('integrations.tasks').setLevel(logging.WARNING)

[Copilot]

The hardcoded file path 'er-prod.gundi-tokens.json' reduces flexibility. Consider making this configurable through command-line arguments or environment variables.

[chrisdoehring]

@vgarcia13 this seems like a good idea.

[Copilot]

The hardcoded file path 'er-prod.vectronic.json' reduces flexibility. Consider making this configurable through command-line arguments or environment variables.

[chrisdoehring]

@vgarcia13 it looks fine. I think it's good to add a cli option for providing the input file.