Skip to content

feat: remove pg_cron dependency and related scheduling logic for improved flexibility #579

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jirhiker merged 2 commits into from
Mar 2, 2026
Merged

feat: remove pg_cron dependency and related scheduling logic for improved flexibility #579

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
98cb0ad
feat: remove pg_cron dependency and related scheduling logic for impr…
jirhiker Mar 2, 2026
0ba9e54
Formatting changes
jirhiker Mar 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 0 additions & 86 deletions alembic/versions/d5e6f7a8b9c0_create_pygeoapi_supporting_views.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,6 @@
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
REFRESH_FUNCTION_NAME = "refresh_pygeoapi_materialized_views"
REFRESH_JOB_NAME = "refresh_pygeoapi_matviews_nightly"
REFRESH_SCHEDULE = "0 3 * * *"

THING_COLLECTIONS = [
("water_wells", "water well"),
Expand Down Expand Up @@ -231,68 +229,6 @@ def _create_refresh_function() -> str:
"""


def _schedule_refresh_job() -> str:
return f"""
DO $do$
BEGIN
BEGIN
-- Avoid direct SELECT on cron.job because managed Postgres
-- environments may deny access to the cron schema table.
PERFORM cron.unschedule('{REFRESH_JOB_NAME}');
EXCEPTION
WHEN undefined_function THEN
NULL;
WHEN invalid_parameter_value THEN
NULL;
WHEN internal_error THEN
-- Some pg_cron builds raise internal_error when the named
-- job does not exist. Treat this as already-unscheduled.
NULL;
WHEN insufficient_privilege THEN
RAISE NOTICE
'Skipping pg_cron unschedule for % due to insufficient privileges.',
'{REFRESH_JOB_NAME}';
RETURN;
END;

PERFORM cron.schedule(
'{REFRESH_JOB_NAME}',
'{REFRESH_SCHEDULE}',
$cmd$SELECT public.{REFRESH_FUNCTION_NAME}();$cmd$
);
EXCEPTION
WHEN insufficient_privilege THEN
RAISE NOTICE
'Skipping pg_cron schedule for % due to insufficient privileges.',
'{REFRESH_JOB_NAME}';
END
$do$;
"""


def _unschedule_refresh_job() -> str:
return f"""
DO $do$
BEGIN
BEGIN
PERFORM cron.unschedule('{REFRESH_JOB_NAME}');
EXCEPTION
WHEN undefined_function THEN
NULL;
WHEN invalid_parameter_value THEN
NULL;
WHEN internal_error THEN
NULL;
WHEN insufficient_privilege THEN
RAISE NOTICE
'Skipping pg_cron unschedule for % due to insufficient privileges.',
'{REFRESH_JOB_NAME}';
END;
END
$do$;
"""


def upgrade() -> None:
bind = op.get_bind()
inspector = inspect(bind)
Expand All @@ -307,16 +243,6 @@ def upgrade() -> None:
f"tables are missing: {missing_tables_str}"
)

pg_cron_available = bind.execute(
text(
"SELECT EXISTS ("
"SELECT 1 FROM pg_available_extensions WHERE name = 'pg_cron'"
")"
)
).scalar()
if pg_cron_available:
op.execute(text("CREATE EXTENSION IF NOT EXISTS pg_cron"))

for view_id, thing_type in THING_COLLECTIONS:
safe_view_id = _safe_view_id(view_id)
op.execute(text(f"DROP VIEW IF EXISTS ogc_{safe_view_id}"))
Expand Down Expand Up @@ -360,21 +286,9 @@ def upgrade() -> None:
_create_matview_indexes()

op.execute(text(_create_refresh_function()))
if pg_cron_available:
op.execute(text(_schedule_refresh_job()))

Comment on lines 286 to 289
Copy link

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This migration previously scheduled a pg_cron job. Editing an existing Alembic revision to remove that logic won’t clean up pg_cron jobs/extensions already present in databases where this revision has been applied, and it changes downgrade behavior for this revision. Prefer adding a new migration that (best-effort) unschedules the old job and drops pg_cron if present, handling privilege errors safely.

Copilot uses AI. Check for mistakes.

def downgrade() -> None:
bind = op.get_bind()
pg_cron_available = bind.execute(
text(
"SELECT EXISTS ("
"SELECT 1 FROM pg_available_extensions WHERE name = 'pg_cron'"
")"
)
).scalar()
if pg_cron_available:
op.execute(text(_unschedule_refresh_job()))
op.execute(text(f"DROP FUNCTION IF EXISTS public.{REFRESH_FUNCTION_NAME}()"))
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Unschedule legacy cron job before dropping refresh function

The updated downgrade() now drops public.refresh_pygeoapi_materialized_views() directly, but earlier versions of this same Alembic revision created a named pg_cron job to call that function nightly. In databases that already applied the old revision, downgrading with this new script leaves that cron entry in place, so pg_cron continues executing a function that no longer exists and generates recurring job failures. Please keep or reintroduce unscheduling logic (or a follow-up migration) for previously migrated environments.

Useful? React with 👍 / 👎.

_drop_view_or_materialized_view("ogc_avg_tds_wells")
_drop_view_or_materialized_view("ogc_latest_depth_to_water_wells")
Expand Down
11 changes: 1 addition & 10 deletions core/initializers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.
# ===============================================================================
from pathlib import Path
import os
from pathlib import Path

from fastapi_pagination import add_pagination
from sqlalchemy import text, select
Expand Down Expand Up @@ -66,15 +66,6 @@ def erase_and_rebuild_db():
session.execute(text("DROP SCHEMA public CASCADE"))
session.execute(text("CREATE SCHEMA public"))
session.execute(text("CREATE EXTENSION IF NOT EXISTS postgis"))
pg_cron_available = session.execute(
text(
"SELECT EXISTS ("
"SELECT 1 FROM pg_available_extensions WHERE name = 'pg_cron'"
")"
)
).scalar()
if pg_cron_available:
session.execute(text("CREATE EXTENSION IF NOT EXISTS pg_cron"))
session.commit()
Base.metadata.drop_all(session.bind)
Base.metadata.create_all(session.bind)
Expand Down
9 changes: 0 additions & 9 deletions db/initialization.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,15 +62,6 @@ def recreate_public_schema(session: Session) -> None:
session.execute(text("DROP SCHEMA public CASCADE"))
session.execute(text("CREATE SCHEMA public"))
session.execute(text("CREATE EXTENSION IF NOT EXISTS postgis"))
pg_cron_available = session.execute(
text(
"SELECT EXISTS ("
"SELECT 1 FROM pg_available_extensions WHERE name = 'pg_cron'"
")"
)
).scalar()
if pg_cron_available:
session.execute(text("CREATE EXTENSION IF NOT EXISTS pg_cron"))
session.execute(APP_READ_GRANT_SQL)
grant_app_read_members(session)
session.commit()
Expand Down
4 changes: 0 additions & 4 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,6 @@ services:
context: .
dockerfile: ./docker/db/Dockerfile
platform: linux/amd64
command: >
postgres
-c shared_preload_libraries=pg_cron
-c cron.database_name=${POSTGRES_DB}
environment:
- POSTGRES_USER=${POSTGRES_USER}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
Expand Down
4 changes: 0 additions & 4 deletions docker/db/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1 @@
FROM postgis/postgis:17-3.5

RUN apt-get update \
&& apt-get install -y --no-install-recommends postgresql-17-cron \
&& rm -rf /var/lib/apt/lists/*