GrowthLens

Customer Value, Targeting and Marketing Investment Optimization Platform

GrowthLens is an independent educational portfolio project built with synthetic data. It is not affiliated with, endorsed by, or based on proprietary systems belonging to any financial institution.

GrowthLens is a local-first, batch analytics MVP for estimating synthetic customer value and conversion propensity, ranking eligible customer-channel opportunities, allocating campaign budgets, measuring ROI, and evaluating randomized experiments. It contains no PII, protected attributes, credit decisions, or real financial data.

Architecture

flowchart LR
  G[Deterministic generator] --> B[Bronze Parquet]
  B --> S[Validation and Silver]
  S --> Q[Quarantine]
  S --> O[Gold serving tables]
  O --> M[Model training and registry]
  M --> R[Targeting recommendations]
  O --> A[Versioned FastAPI]
  R --> A
  A --> D[Seven-view Streamlit dashboard]
  R --> X[CSV and Excel exports]

Loading

The architecture uses PySpark as the primary transformation engine for Bronze → Silver → Gold transformations. Pandas is used for bounded tasks like model training, scoring, optimization, experimentation statistics, drift analysis, and Excel exports. PostgreSQL is the serving layer for the API, with Alembic managing database migrations. An idempotent Gold-to-database loader with row-level reconciliation populates the serving tables.

Quick start

Python 3.12 and uv are required.

uv sync --all-extras
uv run growthlens doctor
uv run growthlens demo --profile ci
uv run uvicorn growthlens.api.main:app --reload
uv run streamlit run src/growthlens/dashboard/app.py

Open API docs at http://localhost:8000/docs and the dashboard at http://localhost:8501. The demo creates Parquet layers, quarantine records, two model artifacts, model cards, a registry, recommendations, experiment results, CSV, and Excel outputs.

Docker

Generate serving artifacts once, then start the read-only services:

docker compose --profile tools run --rm pipeline
docker compose up --build api dashboard postgres

Windows PowerShell

.\scripts\bootstrap.ps1
.\scripts\demo.ps1 -Profile ci
uv run uvicorn growthlens.api.main:app --reload

Linux/macOS Bash

./scripts/bootstrap.sh
./scripts/demo.sh ci

Decision logic

Propensity and value models use activity, digital engagement, service cost, risk-cost proxy, tenure, region, preferred channel, actual channel, recent contacts, and contact cost. Labels and post-outcome values are excluded. The score is converted to:

adjusted_probability = propensity × (1 − probability_haircut)
expected_gross_value = adjusted_probability × predicted_value
expected_incremental_contribution = expected_gross_value × realization_factor × (1 − value_haircut) × incrementality_factor
expected_incentive_cost = adjusted_probability × incentive_cost
expected_total_campaign_cost = contact_cost + expected_incentive_cost
expected_incremental_profit = expected_incremental_contribution − expected_total_campaign_cost
expected_roi = expected_incremental_profit / expected_total_campaign_cost

The deterministic optimizer selects highest-profit eligible rows while enforcing one contact per customer, opt-out suppression, budget, channel capacities, contactability, minimum ROI, and positive profit. This is an auditable greedy heuristic, not an exact mixed-integer solution.

Experiment output separates statistical significance, practical significance, confidence intervals, and sample-ratio mismatch. All outcomes are synthetic and should not be interpreted as business evidence.

Commands

uv run pytest --cov=src/growthlens --cov-report=term-missing
uv run ruff format --check .
uv run ruff check .
uv run mypy src
uv run bandit -r src
uv run pip-audit
docker compose config

Configuration precedence is defaults, base YAML, profile YAML, GROWTHLENS_ environment variables, then CLI arguments. Profiles are ci (50 customers), demo (verified for 1,000 customers), and full (50,000; allow roughly 8–12 GB RAM and additional disk).

Verified capabilities

• Deterministic synthetic data generation (8 PII-free entities, controllable invalid-rate injection)
• PySpark Bronze → Silver → Gold transformation pipeline with schema contracts and temporal normalization
• Silver data-quality enforcement (9 rules) with structured quarantine and pass/fail counts
• Gold customer 360, model training, campaign performance, experiment input, propensity scoring, and targeting products
• Leakage-aware chronological train/test split with whole-campaign periods
• Candidate propensity model selection (LogisticRegression vs HistGradientBoostingClassifier by log loss)
• Customer-value regression model (HistGradientBoostingRegressor on converted customers)
• Model evaluation: ROC-AUC, PR-AUC, Brier, log loss, calibration, lift deciles, offline policy comparison
• Immutable model artifacts, model cards, and lightweight JSON registry
• Transparent economics with Decimal arithmetic, haircuts, incrementality, and explicit ROI warnings
• Greedy constrained optimizer (budget, channel capacity, suppression, contactability, frequency cap, minimum ROI, single contact per customer)
• Five-policy comparison (random, rule-based, propensity-only, value-only, expected-profit)
• A/B testing with z-test, confidence intervals, practical significance, and SRM detection
• PSI-based drift reporting (numerical, categorical, prediction, and missingness)
• Idempotent Gold-to-PostgreSQL loader with SHA-256 fingerprinting and row-level reconciliation
• Alembic-managed migrations with programmatic upgrade/downgrade and health checks
• Typed SQLAlchemy repository layer (795 lines, 18 query methods, pagination-bounded)
• FastAPI with 22 endpoints, RFC 7807 errors, API-key auth, request ID correlation, and liveness/readiness probes
• Seven-view Streamlit dashboard consuming the PostgreSQL-backed API
• Excel export with formula-injection protection, currency formatting, and synthetic-data disclaimer
• Prometheus metrics (HTTP traffic, latency, database failures, scenario duration)
• Docker Compose (PostgreSQL 16, pipeline, API, dashboard) with non-root containers
• CI pipeline: format, lint, type check, Bandit, pip-audit, migration round trip, tests, end-to-end demo

Planned extensions

• Mixed-integer programming optimization
• Individual treatment-effect (uplift) modeling
• Scheduled batch orchestration (Airflow/Dagster)
• Feature store integration
• MLflow or Weights & Biases experiment tracking
• Rate limiting and IAM-integrated authentication

Security, privacy, and limitations

Generated IDs are fictional and non-identifying. Excel strings beginning with formula characters are escaped. Secrets are environment-only. The API defaults to local, unauthenticated demonstration mode and must be placed behind authentication and TLS for any shared environment. Current MVP limitations include heuristic allocation, no causal-uplift claim, no scheduler, and simplified drift reporting. (A file-backed serving fallback exists for local development, but PostgreSQL is the primary serving layer.)

See architecture, modeling, experimentation, operations, security, and the interview guide.