Consolidate enterprise extensions: resources, provisioning, teardown, infra, GCP, CI/CD

[Copilot]

Consolidates PRs #3-#10 into a unified branch. Adds cloud-agnostic resource provisioning, lifecycle management, multi-cloud orchestration (Azure/AWS/GCP/Hetzner), idempotent server provisioning, and CI/CD pipeline generation.

Core Modules

• resources.py - Unified provisioning for databases, cache, queues, storage, LLM endpoints, functions, search across docker/aws/azure/gcp
• connect.py - ResourceConfig for config persistence and Pythonic client wrappers with zero-config defaults
• teardown.py - Idempotent destroy operations with CLI validation
• infra.py - Optional pyinfra integration for idempotent server provisioning
• gcp.py - Cloud Run orchestration with Cloud SQL and Memorystore
• ci.py - GitHub Actions and GitLab CI workflow generation

Ship Orchestrator

Extended with resources parameter for integrated provisioning and Hetzner auto-provisioning via to='hetzner':

from fastops import ship, database, cache

# Zero-config local dev
ship('.', to='docker', resources={
    'db': lambda: database(),      # Postgres with defaults
    'cache': lambda: cache(),      # Redis with defaults
})

# Production with Hetzner auto-provision
ship('.', to='hetzner', 
     domain='api.example.com',
     resources={
         'db': lambda: database(provider='docker'),
         'storage': lambda: bucket('uploads', provider='docker')
     })

Resource Provisioning

Stack returns environment variables, compose config, and volumes. Resources parameter switches providers based on deployment target:

from fastops import stack, ResourceConfig

# Zero-config: Postgres + Redis
env, compose, volumes = stack()

# Export and reconnect
config = ResourceConfig.from_env(env)
config.save('resources.json')
db = config.connect('db')  # Returns fastsql/sqlalchemy client

Dependencies

Added optional extras to pyproject.toml: db, storage, llm, cache, queue, search, azure, aws, infra, all

Updated Exports

__init__.py now exports: database, cache, queue, bucket, llm, function, search, stack, ResourceConfig, destroy, destroy_stack, status, provision, deploy_infra, server_status, gcp_stack, Gcp, gcloud, github_actions, gitlab_ci, deploy_workflow, test_workflow

Statistics

• 7 new modules (~2,700 lines)
• 60+ new functions
• Zero breaking changes

Original prompt

Consolidate All Feature Branches Into One Branch Targeting main

There are currently multiple open PRs (#3 through #10) that form a chain of feature branches. The goal is to consolidate ALL changes from these PRs into a single branch targeting main, so everything can be reviewed and merged in one unified PR.

Current PR Chain Structure:

• PR Add enterprise extensions: proxy builders, auto-detection, compliance, secrets, and multi-cloud orchestration #3 (copilot/implement-enterprise-grade-extensions → main): Enterprise extensions - proxy builders, auto-detection, compliance, secrets, azure/aws CLI wrappers, ship orchestrator (7 new modules)
• PR Add cloud-agnostic resource provisioning layer (databases, caches, queues, storage, LLM, search) #4 (copilot/add-cloud-resource-provisioning → PR Add enterprise extensions: proxy builders, auto-detection, compliance, secrets, and multi-cloud orchestration #3's branch): fastops/resources.py - unified resource provisioning (database, cache, queue, bucket, llm, function, search, stack)
• PR Add ResourceConfig for zero-config resource provisioning and Pythonic client wrappers #5 (copilot/add-resource-config-export → PR Add cloud-agnostic resource provisioning layer (databases, caches, queues, storage, LLM, search) #4's branch): fastops/connect.py - ResourceConfig class, zero-config defaults, Pythonic client wrappers, pyproject.toml optional deps
• PR Add resource teardown, CLI validation, and idempotent provisioning #6 (copilot/add-resource-teardown-methods → PR Add cloud-agnostic resource provisioning layer (databases, caches, queues, storage, LLM, search) #4's branch): fastops/teardown.py - destroy/destroy_stack/status functions, CLI validation, idempotent provisioning
• PR Add ship(to='hetzner') auto-provisioning deployment target #7 (copilot/add-hetzner-deployment-target → PR Add cloud-agnostic resource provisioning layer (databases, caches, queues, storage, LLM, search) #4's branch): ship(to='hetzner') auto-provisioning deployment target in ship.py
• PR Add optional pyinfra integration for idempotent server provisioning #8 (copilot/add-pyinfra-integration → PR Add cloud-agnostic resource provisioning layer (databases, caches, queues, storage, LLM, search) #4's branch): fastops/infra.py - optional pyinfra integration for idempotent server provisioning
• PR Add GCP Cloud Run support with managed services orchestration #9 (GCP Cloud Run support with managed services orchestration, base: PR Add cloud-agnostic resource provisioning layer (databases, caches, queues, storage, LLM, search) #4's branch)
• PR Add CI/CD pipeline generation module for GitHub Actions and GitLab CI #10 (CI/CD pipeline generation module for GitHub Actions and GitLab CI, base: PR Add cloud-agnostic resource provisioning layer (databases, caches, queues, storage, LLM, search) #4's branch)

What To Do:

1. Create a new branch off main (e.g., consolidated/enterprise-extensions)
2. Merge or cherry-pick ALL changes from the following branches (in dependency order):
   • copilot/implement-enterprise-grade-extensions (PR Add enterprise extensions: proxy builders, auto-detection, compliance, secrets, and multi-cloud orchestration #3)
   • copilot/add-cloud-resource-provisioning (PR Add cloud-agnostic resource provisioning layer (databases, caches, queues, storage, LLM, search) #4)
   • copilot/add-resource-config-export (PR Add ResourceConfig for zero-config resource provisioning and Pythonic client wrappers #5)
   • copilot/add-resource-teardown-methods (PR Add resource teardown, CLI validation, and idempotent provisioning #6)
   • copilot/add-hetzner-deployment-target (PR Add ship(to='hetzner') auto-provisioning deployment target #7)
   • copilot/add-pyinfra-integration (PR Add optional pyinfra integration for idempotent server provisioning #8)
   • And branches from PR Add GCP Cloud Run support with managed services orchestration #9 and PR Add CI/CD pipeline generation module for GitHub Actions and GitLab CI #10
3. Resolve any merge conflicts
4. The final branch should contain ALL new modules and changes from every PR, with no conflicts
5. Open a single PR targeting main with a comprehensive description covering all the changes

Expected Files (at minimum):

• fastops/proxy.py - Fluent proxy builders (Caddyfile, NginxConf)
• fastops/detect.py - Framework auto-detection
• fastops/compliance.py - Security & compliance presets
• fastops/secrets.py - Multi-provider secrets management
• fastops/azure.py - Azure CLI wrappers
• fastops/aws.py - AWS CLI wrappers
• fastops/ship.py - Unified orchestrator (with hetzner support)
• fastops/resources.py - Cloud-agnostic resource provisioning (with idempotent provisioning and CLI validation)
• fastops/connect.py - ResourceConfig and client wrappers
• fastops/teardown.py - Resource teardown and lifecycle management
• fastops/infra.py - pyinfra integration for idempotent provisioning
• GCP module from PR Add GCP Cloud Run support with managed services orchestration #9
• CI/CD module from PR Add CI/CD pipeline generation module for GitHub Actions and GitLab CI #10
• Updated fastops/__init__.py with all new exports
• Updated pyproject.toml with optional dependencies

This pull request was created from Copilot chat.

---

🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.