Task S3T07: Create Country Snapshot/Seed Functions¶
Master-Spec Alignment¶
The normative contract for this sprint is 00-SPRINT-INDEX.md plus the canonical Sprint 3 task set in docs/database-refactor/sprint-3/. ../00-SPEC-DRAFT-v1.obsolete.md is legacy context only and is not authoritative.
stats.country_daily_stats.unique_usersandunique_gksare exact online-maintained values, not approximate placeholders.INSERT,UPDATE, andDELETEhandling forstats.gk_countries_visited,stats.user_countries, andstats.gk_country_historymust maintain exact state. When earliest/latest rows are invalidated, affected rows must be recomputed from remaining qualifying moves.- Snapshot functions seed and verify canonical state; they do not compensate for knowingly inexact live maintenance.
- Any lower text that still describes
unique_usersorunique_gksas approximate is obsolete and superseded by this alignment block.
Source¶
- Generated from sprint document step
3.7in03-sprint-3-country-geography.md.
Step 3.7: Create Country Snapshot/Seed Functions¶
What this step does: Creates three idempotent snapshot functions that seed the move-derived country stats state from historical gk_moves data. These functions are used during backfill (Sprint 6) to populate the tables from the ~6.9M existing rows, and can be re-run at any time for correction or verification. All use ON CONFLICT DO UPDATE for idempotency and accept an optional p_period daterange parameter for partial reseeding of the move-derived contract.
Resolved policy for this task:
stats.fn_snapshot_daily_country_stats()owns the move-derived columns instats.country_daily_stats: move counts, per-type counts,unique_users,unique_gks, andkm_contributed.points_contributedremains out of scope for Sprint 3 backfills and stays at the table default (0) until the points phase defines its rebuild contract.loves_countis owned by canonical sourcegeokrety.gk_loves.pictures_uploaded_total,pictures_uploaded_avatar,pictures_uploaded_move, andpictures_uploaded_userare owned by canonical sourcegeokrety.gk_pictures.- Partial
p_periodreseeds leave rows outside the supplied date range untouched. - Full reseed (
p_period IS NULL) is an orchestration concern: rebuild all country stats columns from their canonical sources in sequence, rather than treating the move-derived snapshot as the sole source of truth for non-move columns.
Migration file name: 20260310300600_create_country_snapshot_functions.php
Full SQL DDL¶
-- ============================================================
-- Snapshot 1: Seed country_daily_stats from gk_moves
-- ============================================================
CREATE OR REPLACE FUNCTION stats.fn_snapshot_daily_country_stats(
p_period daterange DEFAULT NULL
)
RETURNS BIGINT
LANGUAGE plpgsql
AS $$
DECLARE
v_count BIGINT := 0;
BEGIN
INSERT INTO stats.country_daily_stats (
stats_date, country_code,
moves_count, drops, grabs, comments, sees, archives, dips,
unique_users, unique_gks, km_contributed
)
SELECT
m.moved_on_datetime::date AS stats_date,
m.country AS country_code,
COUNT(*) AS moves_count,
COUNT(*) FILTER (WHERE m.move_type = 0) AS drops,
COUNT(*) FILTER (WHERE m.move_type = 1) AS grabs,
COUNT(*) FILTER (WHERE m.move_type = 2) AS comments,
COUNT(*) FILTER (WHERE m.move_type = 3) AS sees,
COUNT(*) FILTER (WHERE m.move_type = 4) AS archives,
COUNT(*) FILTER (WHERE m.move_type = 5) AS dips,
COUNT(DISTINCT m.author) FILTER (WHERE m.author IS NOT NULL) AS unique_users,
COUNT(DISTINCT m.geokret) AS unique_gks,
COALESCE(SUM(m.km_distance), 0) AS km_contributed
FROM geokrety.gk_moves m
WHERE m.country IS NOT NULL
AND (p_period IS NULL OR m.moved_on_datetime::date <@ p_period)
GROUP BY m.moved_on_datetime::date, m.country
ON CONFLICT (stats_date, country_code) DO UPDATE SET
moves_count = EXCLUDED.moves_count,
drops = EXCLUDED.drops,
grabs = EXCLUDED.grabs,
comments = EXCLUDED.comments,
sees = EXCLUDED.sees,
archives = EXCLUDED.archives,
dips = EXCLUDED.dips,
unique_users = EXCLUDED.unique_users,
unique_gks = EXCLUDED.unique_gks,
km_contributed = EXCLUDED.km_contributed;
GET DIAGNOSTICS v_count = ROW_COUNT;
RETURN v_count;
END;
$$;
COMMENT ON FUNCTION stats.fn_snapshot_daily_country_stats IS 'Seeds country_daily_stats from gk_moves. Idempotent via ON CONFLICT DO UPDATE. Optional p_period limits date range.';
-- ============================================================
-- Snapshot 2: Seed user_countries from gk_moves
-- ============================================================
CREATE OR REPLACE FUNCTION stats.fn_snapshot_user_country_stats(
p_period daterange DEFAULT NULL
)
RETURNS BIGINT
LANGUAGE plpgsql
AS $$
DECLARE
v_count BIGINT := 0;
BEGIN
INSERT INTO stats.user_countries (
user_id, country_code, move_count, first_visit, last_visit
)
SELECT
m.author AS user_id,
m.country AS country_code,
COUNT(*) AS move_count,
MIN(m.moved_on_datetime) AS first_visit,
MAX(m.moved_on_datetime) AS last_visit
FROM geokrety.gk_moves m
WHERE m.country IS NOT NULL
AND m.author IS NOT NULL
AND (p_period IS NULL OR m.moved_on_datetime::date <@ p_period)
GROUP BY m.author, m.country
ON CONFLICT (user_id, country_code) DO UPDATE SET
move_count = EXCLUDED.move_count,
first_visit = LEAST(stats.user_countries.first_visit, EXCLUDED.first_visit),
last_visit = GREATEST(stats.user_countries.last_visit, EXCLUDED.last_visit);
GET DIAGNOSTICS v_count = ROW_COUNT;
RETURN v_count;
END;
$$;
COMMENT ON FUNCTION stats.fn_snapshot_user_country_stats IS 'Seeds user_countries from gk_moves. Idempotent via ON CONFLICT DO UPDATE. Optional p_period limits date range.';
-- ============================================================
-- Snapshot 3: Seed gk_countries_visited from gk_moves
-- ============================================================
CREATE OR REPLACE FUNCTION stats.fn_snapshot_gk_country_stats(
p_period daterange DEFAULT NULL
)
RETURNS BIGINT
LANGUAGE plpgsql
AS $$
DECLARE
v_count BIGINT := 0;
BEGIN
INSERT INTO stats.gk_countries_visited (
geokrety_id, country_code, first_visited_at, first_move_id, move_count
)
SELECT
sub.geokret AS geokrety_id,
sub.country AS country_code,
sub.first_visited_at,
sub.first_move_id,
sub.move_count
FROM (
SELECT
m.geokret,
m.country,
MIN(m.moved_on_datetime) AS first_visited_at,
(array_agg(m.id ORDER BY m.moved_on_datetime ASC, m.id ASC))[1] AS first_move_id,
COUNT(*) AS move_count
FROM geokrety.gk_moves m
WHERE m.country IS NOT NULL
AND (p_period IS NULL OR m.moved_on_datetime::date <@ p_period)
GROUP BY m.geokret, m.country
) sub
ON CONFLICT (geokrety_id, country_code) DO UPDATE SET
move_count = EXCLUDED.move_count,
first_visited_at = LEAST(stats.gk_countries_visited.first_visited_at, EXCLUDED.first_visited_at),
first_move_id = CASE
WHEN EXCLUDED.first_visited_at < stats.gk_countries_visited.first_visited_at THEN EXCLUDED.first_move_id
ELSE stats.gk_countries_visited.first_move_id
END;
GET DIAGNOSTICS v_count = ROW_COUNT;
RETURN v_count;
END;
$$;
COMMENT ON FUNCTION stats.fn_snapshot_gk_country_stats IS 'Seeds gk_countries_visited from gk_moves. Idempotent via ON CONFLICT DO UPDATE. Optional p_period limits date range.';
Full Phinx Migration PHP Code¶
<?php
declare(strict_types=1);
use Phinx\Migration\AbstractMigration;
final class CreateCountrySnapshotFunctions extends AbstractMigration
{
public function up(): void
{
$this->execute(<<<'SQL'
CREATE OR REPLACE FUNCTION stats.fn_snapshot_daily_country_stats(
p_period daterange DEFAULT NULL
)
RETURNS BIGINT
LANGUAGE plpgsql
AS $$
DECLARE
v_count BIGINT := 0;
BEGIN
INSERT INTO stats.country_daily_stats (
stats_date, country_code,
moves_count, drops, grabs, comments, sees, archives, dips,
unique_users, unique_gks, km_contributed
)
SELECT
m.moved_on_datetime::date AS stats_date,
m.country AS country_code,
COUNT(*) AS moves_count,
COUNT(*) FILTER (WHERE m.move_type = 0) AS drops,
COUNT(*) FILTER (WHERE m.move_type = 1) AS grabs,
COUNT(*) FILTER (WHERE m.move_type = 2) AS comments,
COUNT(*) FILTER (WHERE m.move_type = 3) AS sees,
COUNT(*) FILTER (WHERE m.move_type = 4) AS archives,
COUNT(*) FILTER (WHERE m.move_type = 5) AS dips,
COUNT(DISTINCT m.author) FILTER (WHERE m.author IS NOT NULL) AS unique_users,
COUNT(DISTINCT m.geokret) AS unique_gks,
COALESCE(SUM(m.km_distance), 0) AS km_contributed
FROM geokrety.gk_moves m
WHERE m.country IS NOT NULL
AND (p_period IS NULL OR m.moved_on_datetime::date <@ p_period)
GROUP BY m.moved_on_datetime::date, m.country
ON CONFLICT (stats_date, country_code) DO UPDATE SET
moves_count = EXCLUDED.moves_count,
drops = EXCLUDED.drops,
grabs = EXCLUDED.grabs,
comments = EXCLUDED.comments,
sees = EXCLUDED.sees,
archives = EXCLUDED.archives,
dips = EXCLUDED.dips,
unique_users = EXCLUDED.unique_users,
unique_gks = EXCLUDED.unique_gks,
km_contributed = EXCLUDED.km_contributed;
GET DIAGNOSTICS v_count = ROW_COUNT;
RETURN v_count;
END;
$$;
COMMENT ON FUNCTION stats.fn_snapshot_daily_country_stats IS 'Seeds country_daily_stats from gk_moves. Idempotent via ON CONFLICT DO UPDATE. Optional p_period limits date range.';
CREATE OR REPLACE FUNCTION stats.fn_snapshot_user_country_stats(
p_period daterange DEFAULT NULL
)
RETURNS BIGINT
LANGUAGE plpgsql
AS $$
DECLARE
v_count BIGINT := 0;
BEGIN
INSERT INTO stats.user_countries (
user_id, country_code, move_count, first_visit, last_visit
)
SELECT
m.author AS user_id,
m.country AS country_code,
COUNT(*) AS move_count,
MIN(m.moved_on_datetime) AS first_visit,
MAX(m.moved_on_datetime) AS last_visit
FROM geokrety.gk_moves m
WHERE m.country IS NOT NULL
AND m.author IS NOT NULL
AND (p_period IS NULL OR m.moved_on_datetime::date <@ p_period)
GROUP BY m.author, m.country
ON CONFLICT (user_id, country_code) DO UPDATE SET
move_count = EXCLUDED.move_count,
first_visit = LEAST(stats.user_countries.first_visit, EXCLUDED.first_visit),
last_visit = GREATEST(stats.user_countries.last_visit, EXCLUDED.last_visit);
GET DIAGNOSTICS v_count = ROW_COUNT;
RETURN v_count;
END;
$$;
COMMENT ON FUNCTION stats.fn_snapshot_user_country_stats IS 'Seeds user_countries from gk_moves. Idempotent via ON CONFLICT DO UPDATE. Optional p_period limits date range.';
CREATE OR REPLACE FUNCTION stats.fn_snapshot_gk_country_stats(
p_period daterange DEFAULT NULL
)
RETURNS BIGINT
LANGUAGE plpgsql
AS $$
DECLARE
v_count BIGINT := 0;
BEGIN
INSERT INTO stats.gk_countries_visited (
geokrety_id, country_code, first_visited_at, first_move_id, move_count
)
SELECT
sub.geokret AS geokrety_id,
sub.country AS country_code,
sub.first_visited_at,
sub.first_move_id,
sub.move_count
FROM (
SELECT
m.geokret,
m.country,
MIN(m.moved_on_datetime) AS first_visited_at,
(array_agg(m.id ORDER BY m.moved_on_datetime ASC, m.id ASC))[1] AS first_move_id,
COUNT(*) AS move_count
FROM geokrety.gk_moves m
WHERE m.country IS NOT NULL
AND (p_period IS NULL OR m.moved_on_datetime::date <@ p_period)
GROUP BY m.geokret, m.country
) sub
ON CONFLICT (geokrety_id, country_code) DO UPDATE SET
move_count = EXCLUDED.move_count,
first_visited_at = LEAST(stats.gk_countries_visited.first_visited_at, EXCLUDED.first_visited_at),
first_move_id = CASE
WHEN EXCLUDED.first_visited_at < stats.gk_countries_visited.first_visited_at THEN EXCLUDED.first_move_id
ELSE stats.gk_countries_visited.first_move_id
END;
GET DIAGNOSTICS v_count = ROW_COUNT;
RETURN v_count;
END;
$$;
COMMENT ON FUNCTION stats.fn_snapshot_gk_country_stats IS 'Seeds gk_countries_visited from gk_moves. Idempotent via ON CONFLICT DO UPDATE. Optional p_period limits date range.';
SQL
);
}
public function down(): void
{
$this->execute('DROP FUNCTION IF EXISTS stats.fn_snapshot_daily_country_stats(daterange);');
$this->execute('DROP FUNCTION IF EXISTS stats.fn_snapshot_user_country_stats(daterange);');
$this->execute('DROP FUNCTION IF EXISTS stats.fn_snapshot_gk_country_stats(daterange);');
}
}
SQL Usage Examples¶
-- Full snapshot of all country daily stats (all time)
SELECT stats.fn_snapshot_daily_country_stats(NULL);
-- Snapshot only for January 2025
SELECT stats.fn_snapshot_daily_country_stats('[2025-01-01, 2025-02-01)'::daterange);
-- Snapshot user country stats for all time
SELECT stats.fn_snapshot_user_country_stats(NULL);
-- Snapshot GK country stats for a specific quarter
SELECT stats.fn_snapshot_gk_country_stats('[2025-01-01, 2025-04-01)'::daterange);
-- Verify idempotency: run twice, check row counts match
SELECT stats.fn_snapshot_daily_country_stats('[2025-06-01, 2025-07-01)'::daterange) AS first_run;
SELECT stats.fn_snapshot_daily_country_stats('[2025-06-01, 2025-07-01)'::daterange) AS second_run;
-- Both return same count; ON CONFLICT DO UPDATE overwrites identical values
Graph/Visualization Specification¶
No new visualization unlocked at this step. These functions populate data for the visualizations defined in Steps 3.1–3.4.
TimescaleDB Assessment¶
NOT applicable. This step creates functions, not tables.
pgTAP Unit Tests¶
| Test ID | Test Name | Assertion |
|---|---|---|
| T-3.7.001 | fn_snapshot_daily_country_stats exists | has_function('stats', 'fn_snapshot_daily_country_stats', ARRAY['daterange']) |
| T-3.7.002 | fn_snapshot_user_country_stats exists | has_function('stats', 'fn_snapshot_user_country_stats', ARRAY['daterange']) |
| T-3.7.003 | fn_snapshot_gk_country_stats exists | has_function('stats', 'fn_snapshot_gk_country_stats', ARRAY['daterange']) |
| T-3.7.004 | fn_snapshot_daily_country_stats returns bigint | function_returns('stats', 'fn_snapshot_daily_country_stats', ARRAY['daterange'], 'bigint') |
| T-3.7.005 | fn_snapshot_user_country_stats returns bigint | function_returns('stats', 'fn_snapshot_user_country_stats', ARRAY['daterange'], 'bigint') |
| T-3.7.006 | fn_snapshot_gk_country_stats returns bigint | function_returns('stats', 'fn_snapshot_gk_country_stats', ARRAY['daterange'], 'bigint') |
| T-3.7.007 | Snapshot daily produces correct row count | Insert 3 test moves (2 in PL, 1 in DE), run snapshot, verify 2 rows in country_daily_stats |
| T-3.7.008 | Snapshot is idempotent | Run snapshot twice on same data, verify same row count both times |
| T-3.7.009 | Snapshot with p_period filters correctly | Insert moves in Jan and Feb, snapshot for Jan only, verify only Jan rows populated |
| T-3.7.010 | User snapshot skips anonymous moves | Insert move with author=NULL, run user snapshot, verify no row for NULL user |
| T-3.7.011 | GK snapshot captures first_move_id correctly | Insert 3 moves for GK #1 in PL with different IDs, verify first_move_id is the earliest |
Implementation Checklist¶
- 1. Create migration file
20260310300600_create_country_snapshot_functions.php - 2. Run
phinx migrate - 3. Verify all three functions exist with correct signatures
- 4. Test each function with NULL period (full snapshot)
- 5. Test each function with a specific date range
- 6. Test idempotency: run twice with same data
- 7. Test that anonymous moves are excluded from user snapshot
- 8. Run pgTAP tests T-3.7.001 through T-3.7.011
Agent Loop Log¶
- 2026-03-10T18:16:06Z - Loop 1 -
dba: Confirmed the existing SQL should remain move-derived only; non-move columns need separate canonical-source rebuild ownership instead of being silently zeroed by this function. - 2026-03-10T18:16:06Z - Loop 1 -
critical-thinking: No blocking contradiction; clarified that partial reseeds protect out-of-range rows while full reseeds are orchestration-level rebuilds across all owning sources. - 2026-03-10T18:16:06Z - Loop 1 -
specification: Folded the ownership and cleanup policy into the task text, removed the open-question placeholders, and preserved the existing snapshot SQL scope.
Resolution¶
- Backfill ownership and cleanup policy are canonicalized in this file; see Q-025 reference update in
99-OPEN-QUESTIONS.md.