Skip to Content
Migration Plan

Migration Plan: DermaDetect Monorepo

Overview

This document outlines the detailed migration strategy for consolidating four separate codebases into a unified FastAPI-based backend_2025.

Source Projects:

  1. algo_python - Flask-based AI/ML service (193 Python files)
  2. dd_api - NestJS-based API gateway (47 TypeScript files)
  3. api-mobile-maccabi - NestJS-based mobile API (25 TypeScript files)
  4. api-backoffice-maccabi - NestJS-based physician portal (40 TypeScript files)

Target Structure:

  • services/ai_service/ - FastAPI (from algo_python)
  • services/api_gateway/ - FastAPI (from dd_api + mobile + backoffice)
  • packages/py_core/ - Shared utilities

Phase 1: Foundation & Tooling (Week 1)

1.1 Monorepo Structure Setup

Tasks:

  • Create directory structure matching brain-stem pattern
  • Initialize Git repository
  • Set up root pyproject.toml with workspace configuration
  • Create justfile for common commands
  • Set up .gitignore, .pre-commit-config.yaml
  • Configure ruff linting rules
  • Create docker-compose.yml for local development

Deliverables:

backend_2025/
├── packages/py_core/
├── services/ai_service/
├── services/api_gateway/
├── docs/
├── integration_tests/
├── docker/
├── scripts/
├── pyproject.toml
├── justfile
├── docker-compose.yml
├── .gitignore
├── .pre-commit-config.yaml
└── README.md

1.2 Shared Core Package Setup

Tasks:

  • Create packages/py_core/pyproject.toml
  • Set up basic package structure:
    • src/auth/ - JWT utilities, password hashing
    • src/db/ - Database connection management
    • src/models/ - Pydantic base models
    • src/repositories/ - Base repository patterns
    • src/logging_config.py - Structured logging setup
    • src/storage/ - Cloud storage abstractions
    • src/utils/ - Common utilities
  • Add dependencies: FastAPI, SQLAlchemy, asyncpg, pydantic, structlog
  • Create test structure
  • Set up pytest configuration

Key Files:

  • packages/py_core/src/auth/jwt.py - JWT encoding/decoding
  • packages/py_core/src/auth/password.py - Password hashing (passlib)
  • packages/py_core/src/db/session.py - Async database session
  • packages/py_core/src/db/base.py - SQLAlchemy base class
  • packages/py_core/src/logging_config.py - Structlog configuration
  • packages/py_core/src/storage/base.py - Abstract storage interface
  • packages/py_core/src/storage/azure.py - Azure Blob Storage
  • packages/py_core/src/storage/gcs.py - Google Cloud Storage

1.3 Database Schema Design

Tasks:

  • Analyze existing schemas from:
    • dd_api: prisma/schema.prisma
    • api-mobile-maccabi: TypeORM entities
    • api-backoffice-maccabi: TypeORM entities
  • Design unified schema in SQLAlchemy
  • Create Alembic migration setup
  • Create initial migration with all tables
  • Create Pydantic models for all entities

Key Tables:

  • tenants - Multi-tenant support
  • users - Patient accounts
  • physicians - Physician accounts
  • rn_requests - Patient cases
  • dermatologist_annotation - Diagnoses
  • diseases - Disease master list
  • templates - Diagnosis templates
  • service_log - API logging
  • audit_annotation - Audit trail

Files:

  • packages/py_core/src/models/tenant.py
  • packages/py_core/src/models/user.py
  • packages/py_core/src/models/physician.py
  • packages/py_core/src/models/case.py
  • packages/py_core/src/models/annotation.py
  • packages/py_core/src/models/disease.py
  • packages/py_core/src/models/template.py

Phase 2: AI Service Migration (Weeks 2-4)

2.1 Service Structure Setup

Tasks:

  • Create services/ai_service/pyproject.toml
  • Set up FastAPI application structure:
    • src/main.py - FastAPI app initialization
    • src/api/ - API route handlers
    • src/core/ - Business logic (from algo_python/src/core)
    • src/config.py - Service configuration
  • Add py_core as local dependency
  • Create Dockerfile

Directory Mapping:

algo_python/src/core/          → services/ai_service/src/core/
algo_python/src/api/MyDerma/   → services/ai_service/src/api/v1/
algo_python/src/protocols/     → services/ai_service/src/schemas/
algo_python/src/connect/       → packages/py_core/src/storage/

2.2 Core Logic Migration

Priority 1 - Critical Endpoints:

  1. Diagnosis Prediction (/ai/v1/diagnosis/predict)

    • Source: algo_python/src/core/diagnostics/predictor_manager.py
    • Target: services/ai_service/src/core/diagnostics/predictor.py
    • Tasks:
      • Convert to async functions
      • Create Pydantic request/response models
      • Migrate Gen2API predictor logic
      • Migrate MedicalSupervisor post-processing
      • Add FastAPI route handler
      • Write unit tests

  2. Red Flags Detection (/ai/v1/diagnosis/red_flags)

    • Source: algo_python/src/core/red_flags/red_flags.py
    • Target: services/ai_service/src/core/red_flags/detector.py
    • Tasks:
      • Migrate red flags logic
      • Create Pydantic models
      • Add FastAPI route
      • Write tests

  3. Image Quality Validation (/ai/v1/image_quality/validate)

    • Source: algo_python/src/core/image_quality/image_quality_control.py
    • Target: services/ai_service/src/core/image_quality/validator.py
    • Tasks:
      • Migrate image validation logic
      • Add async image download from cloud
      • Create Pydantic models
      • Add FastAPI route
      • Write tests

  4. Next Question (/ai/v1/questionnaire/next)

    • Source: algo_python/src/core/next_question/next_question_manager.py
    • Target: services/ai_service/src/core/questionnaire/manager.py
    • Tasks:
      • Migrate question flow logic
      • Load question_flow.csv
      • Create Pydantic models
      • Add FastAPI route
      • Write tests

  5. Triage Assessment (/ai/v1/diagnosis/triage)

    • Source: algo_python/src/api/triage/triage_api.py
    • Target: services/ai_service/src/core/triage/assessor.py
    • Tasks:
      • Migrate triage logic
      • Create Pydantic models
      • Add FastAPI route
      • Write tests

Priority 2 - Supporting Components:

  1. Deep Learning Models

    • Source: algo_python/src/core/deep/
    • Target: services/ai_service/src/core/models/
    • Tasks:
      • Migrate model loading logic
      • Set up model caching
      • Add async model inference
      • Create model registry

  2. Data Processing

    • Source: algo_python/src/protocols/patient_request.py
    • Target: services/ai_service/src/schemas/patient_request.py
    • Tasks:
      • Convert to Pydantic models
      • Add validation rules
      • Migrate data transformation logic

2.3 Dependencies Migration

Python Packages to Migrate:

  • Flask → FastAPI
  • gunicorn → uvicorn
  • (Keep) tensorflow, scikit-learn, opencv-python, pandas, numpy
  • Add: httpx (for async HTTP), aiofiles (for async file I/O)

Configuration:

  • Migrate INI-based logging → structlog configuration
  • Environment variables → Pydantic Settings
  • Azure/GCS credentials → py_core storage module

2.4 Testing

Tasks:

  • Set up pytest with async support
  • Create test fixtures for models
  • Write unit tests for each core module
  • Create integration tests for API endpoints
  • Add test coverage reporting

Test Structure:

services/ai_service/src/
  api/
    v1/
      diagnosis_test.py
      image_quality_test.py
      questionnaire_test.py
  core/
    diagnostics/
      predictor_test.py
    red_flags/
      detector_test.py

Phase 3: API Gateway Migration (Weeks 5-8)

3.1 Service Structure Setup

Tasks:

  • Create services/api_gateway/pyproject.toml
  • Set up FastAPI application structure:
    • src/main.py - FastAPI app with multiple routers
    • src/api/v1/ - Versioned API routes
    • src/auth/ - Authentication module
    • src/services/ - Business logic services
    • src/schemas/ - Pydantic request/response models
    • src/config.py - Service configuration
  • Add py_core as local dependency
  • Create Dockerfile

Directory Structure:

services/api_gateway/src/
├── main.py
├── config.py
├── api/
│   └── v1/
│       ├── auth.py
│       ├── cases.py
│       ├── annotations.py
│       ├── templates.py
│       ├── images.py
│       ├── mobile.py
│       └── backoffice.py
├── auth/
│   ├── jwt.py
│   ├── guards.py
│   └── tenants.py
├── services/
│   ├── case_service.py
│   ├── annotation_service.py
│   ├── ai_service.py (HTTP client to AI service)
│   └── storage_service.py
└── schemas/
    ├── auth.py
    ├── case.py
    ├── annotation.py
    └── mobile.py

3.2 Authentication Module Migration

Source: dd_api/src/auth/ + api-backoffice-maccabi/src/auth/

Tasks:

  1. JWT Authentication

    • Source: dd_api/src/auth/auth.guard.ts, api-backoffice-maccabi/src/auth/jwt.strategy.ts
    • Target: services/api_gateway/src/auth/jwt.py
    • Create FastAPI Depends() for JWT validation
    • Support multiple JWT payload formats (Maccabi, B2C, etc.)
    • Add user lookup after token validation

  2. Local Authentication

    • Source: dd_api/src/auth/auth.service.ts, api-backoffice-maccabi/src/auth/local.strategy.ts
    • Target: services/api_gateway/src/auth/local.py
    • Email/password validation
    • Password hashing with passlib
    • Token generation

  3. SAML/SSO Authentication (Backoffice only)

    • Source: api-backoffice-maccabi/src/saml/
    • Target: services/api_gateway/src/auth/saml.py
    • SAML 2.0 integration
    • Active Directory support
    • Auto-provisioning users

  4. Tenant Management

    • Source: dd_api/src/auth/tenants/, api-mobile-maccabi/src/shared/general/
    • Target: services/api_gateway/src/auth/tenants.py
    • Tenant validation from URL path
    • API key authentication
    • Request counting middleware

3.3 Patient API Migration (from dd_api)

Base Path: /api/v1/

Endpoints to Migrate:

  1. Authentication (/api/v1/auth/)

    • Source: dd_api/src/auth/auth.controller.ts
    • POST /login - User login
    • POST /register - User registration
    • POST /token-validity - Validate JWT

  2. AI Proxy (/api/v1/ai/)

    • Source: dd_api/src/coreAPI/artificial-intelligence/
    • POST /predict_diagnosis - Proxy to AI service
    • POST /red_flags - Proxy to AI service
    • POST /image_validity - Proxy to AI service
    • POST /triage - Proxy to AI service
    • POST /next_question - Proxy to AI service
    • Add translation layer (4 languages)
    • Add service logging
    • Save cases to database

  3. Image Management (/api/v1/images/)

    • Source: dd_api/src/coreAPI/common/common.controller.ts
    • POST /upload - Upload image to cloud storage
    • GET /:path - Download image (authenticated)

  4. Frontend Utilities (/api/v1/frontend/)

    • Source: dd_api/src/coreAPI/common/frontend/
    • GET /download_resource/:name - Download resource
    • POST /user_selection_to_primary_body_location - Convert coordinates

  5. Reports (/api/v1/reports/)

    • Source: dd_api/src/coreAPI/reports/
    • POST /get_pdf - Generate PDF report

3.4 Mobile API Migration (from api-mobile-maccabi)

Base Path: /api/v1/mobile/

Endpoints to Migrate:

  1. Case Management (/api/v1/mobile/cases/)

    • Source: api-mobile-maccabi/src/maccabi/maccabi.controller.ts
    • POST /create - Create Maccabi case
    • POST /update - Update case with patient details
    • POST /:uuid/status - Update case status (viewed)
    • GET /:uuid/annotations - Get annotations

  2. User Management (/api/v1/mobile/users/)

    • Source: api-mobile-maccabi/src/shared/general/general.controller.ts
    • POST /offsprings - Get family members (Maccabi API)
    • POST /offsprings-v2/:technicalID - Get family members v2
    • DELETE /:userId - Delete user data (GDPR)

  3. Image Management (/api/v1/mobile/images/)

    • Source: api-mobile-maccabi/src/shared/general/general.controller.ts
    • POST /upload - Upload to Firebase
    • DELETE /:name - Delete from Firebase

  4. Configuration (/api/v1/mobile/config/)

    • Source: api-mobile-maccabi/src/shared/general/, src/shared/parameter/
    • POST /check-connectivity - Version check
    • GET /parameters/:branch/:sub_branch - Get translations

  5. Utilities (/api/v1/mobile/)

    • Source: api-mobile-maccabi/src/shared/mailer/
    • POST /send-email - Send contact email

3.5 Backoffice API Migration (from api-backoffice-maccabi)

Base Path: /api/v1/backoffice/

Endpoints to Migrate:

  1. Authentication (/api/v1/backoffice/auth/)

    • Source: api-backoffice-maccabi/src/auth/auth.controller.ts
    • POST /login - Physician login
    • POST /logout - Logout
    • POST /is-auth - Check authentication
    • POST /gen-access-token - Generate API token
    • POST /invitation - Invite new physician
    • POST /registration-validation - Validate registration
    • POST /registration-password - Set password
    • PATCH /forgot-password - Reset password
    • GET /csrf-token - Get CSRF token
    • GET /env-settings - Get environment settings

  2. SAML Authentication (/api/v1/backoffice/saml/)

    • Source: api-backoffice-maccabi/src/saml/
    • GET /metadata - SAML metadata
    • GET /login - SAML login
    • POST /assert - SAML assertion
    • GET /logout - SAML logout
    • GET /is-auth - Check SAML auth

  3. Case Management (/api/v1/backoffice/cases/)

    • Source: api-backoffice-maccabi/src/request/
    • POST /next-case - Get next available case (NAC algorithm)
    • POST /search - Search cases with filters
    • GET /:uuid - Get case details
    • POST /:uuid/status - Update case status
    • POST /:uuid/update-record - Update record details
    • GET /:uuid/pdf - Get PDF report
    • POST /check-batch - Check if case has batch
    • GET /batch/:uuid - Get all cases in batch
    • PATCH /changes-pending-status - Block/release batch
    • PATCH /notify-patient - Notify patient
    • POST /:uuid/trigger-eligibility - Trigger eligibility check

  4. Physician Management (/api/v1/backoffice/physicians/)

    • Source: api-backoffice-maccabi/src/doctor/
    • POST / - Create physician
    • POST /details - Get physician details
    • PATCH /settings - Update physician settings
    • PATCH /activity - Activate/deactivate physician

  5. Annotations (/api/v1/backoffice/annotations/)

    • Source: api-backoffice-maccabi/src/dermatologist-annotation/
    • POST / - Get annotations by UUIDs (bulk)
    • GET /:uuid - Get annotations by case UUID
    • POST /:uuid - Create annotation
    • PUT /:uuid - Update annotation
    • PUT /:uuid/cancel - Cancel annotation
    • POST /:uuid/notify - Notify patient

  6. Templates (/api/v1/backoffice/templates/)

    • Source: api-backoffice-maccabi/src/templates/
    • GET / - Get all templates
    • GET /disease/:diseaseId - Get templates by disease
    • GET /doctor/:email - Get templates by physician
    • POST / - Create template
    • PATCH /:id - Update template
    • DELETE /:id - Delete template

  7. Diseases (/api/v1/backoffice/diseases/)

    • Source: api-backoffice-maccabi/src/diseases/
    • GET /:vendor - Get diseases by vendor

  8. Images (/api/v1/backoffice/images/)

    • Source: api-backoffice-maccabi/src/
    • GET / - Get image by hash (Firebase)
    • GET /:imgid - Get case image (GCP/Azure)

  9. Utilities (/api/v1/backoffice/)

    • Source: api-backoffice-maccabi/src/common/
    • GET /history/:user_identifier - Get patient history
    • POST /format-diagnoses - Format diagnosis data
    • POST /set-language - Set physician language

  10. File Import (/api/v1/backoffice/file-import/)

    • Source: api-backoffice-maccabi/src/file-import/
    • POST /upload - Upload Excel for bulk import

  11. WebSocket/SSE (/api/v1/backoffice/)

    • Source: api-backoffice-maccabi/src/requests/, src/unique-sessions/
    • WebSocket /subscribe-analytics - Case analytics updates
    • SSE /unique-sessions/connect - Single session enforcement

3.6 Service Layer

Tasks:

  1. Case Service

    • Target: services/api_gateway/src/services/case_service.py
    • Case creation with priority assignment
    • Case update logic
    • Case search with filters
    • NAC (Next Available Case) algorithm
    • Batch operations
    • Status transitions

  2. Annotation Service

    • Target: services/api_gateway/src/services/annotation_service.py
    • Create/update annotations
    • Validate physician ownership
    • Notification logic
    • Audit trail

  3. AI Service Client

    • Target: services/api_gateway/src/services/ai_service.py
    • HTTP client to AI service (httpx)
    • Retry logic
    • Translation integration
    • Service logging

  4. Storage Service

    • Target: services/api_gateway/src/services/storage_service.py
    • Upload images to cloud
    • Download images from cloud
    • Support multiple providers (Azure, GCS, Firebase)
    • Image validation

  5. External Integration Service

    • Target: services/api_gateway/src/services/external_service.py
    • Maccabi API integration (offsprings)
    • Eligibility API integration
    • Email service (Mailgun)
    • SMS service (Twilio)

3.7 Testing

Tasks:

  • Set up pytest with async support
  • Create test database (Docker PostgreSQL)
  • Mock AI service for integration tests
  • Mock cloud storage for tests
  • Create test fixtures for auth
  • Write unit tests for each service
  • Write integration tests for each endpoint
  • Add test coverage reporting (target: >80%)

Phase 4: Integration & Testing (Week 9-10)

4.1 Integration Testing

Tasks:

  • Set up integration test environment
  • Create Docker Compose for test stack
  • Write end-to-end tests:
    • Patient case submission flow
    • Physician diagnosis flow
    • Mobile app flow (Maccabi)
    • Backoffice admin flow
  • Test multi-tenancy isolation
  • Test authentication flows
  • Test cloud storage operations

Test Structure:

integration_tests/
├── conftest.py
├── test_patient_flow.py
├── test_physician_flow.py
├── test_mobile_flow.py
├── test_backoffice_flow.py
└── test_auth_flows.py

4.2 Performance Testing

Tasks:

  • Set up load testing (locust or k6)
  • Test API Gateway throughput
  • Test AI Service latency
  • Database query optimization
  • Identify bottlenecks
  • Optimize async operations

4.3 Security Testing

Tasks:

  • Test JWT validation
  • Test multi-tenant isolation
  • Test rate limiting
  • Test input validation
  • Test SQL injection prevention
  • Test XSS prevention
  • Run security scanner (bandit)

Phase 5: Deployment & Documentation (Week 11-12)

5.1 Docker & Deployment

Tasks:

  • Optimize Dockerfiles for production
  • Create docker-compose for production
  • Set up health checks
  • Configure logging aggregation
  • Set up Prometheus metrics
  • Create Kubernetes manifests (optional)

5.2 Documentation

Tasks:

  • Complete API documentation (OpenAPI/Swagger)
  • Update all README files
  • Create deployment guide
  • Create developer onboarding guide
  • Document environment variables
  • Create troubleshooting guide
  • Add diagrams (architecture, data flow, etc.)

5.3 Migration Guide

Tasks:

  • Document breaking changes
  • Create client migration guide
  • Document new API endpoints
  • Create comparison table (old vs new)
  • Document rollback procedures

5.4 Cutover Planning

Tasks:

  • Create cutover checklist
  • Plan blue-green deployment strategy
  • Set up feature flags
  • Plan gradual rollout
  • Create monitoring dashboards
  • Document rollback procedures

Success Criteria

Functional Requirements

  • ✅ All endpoints from 4 source projects are migrated
  • ✅ All business logic is preserved
  • ✅ All integrations work (Maccabi API, cloud storage, etc.)
  • ✅ Multi-tenancy is fully functional
  • ✅ Authentication/authorization works for all user types

Non-Functional Requirements

  • ✅ API response time < 500ms (p95)
  • ✅ AI prediction time < 3 seconds (p95)
  • ✅ Database queries < 100ms (p95)
  • ✅ Test coverage > 80%
  • ✅ No security vulnerabilities (critical/high)
  • ✅ Comprehensive documentation

Quality Metrics

  • ✅ All unit tests passing
  • ✅ All integration tests passing
  • ✅ Linting passes (ruff)
  • ✅ Type checking passes (mypy, optional)
  • ✅ Security scan passes (bandit)

Risk Management

High-Risk Items

  1. ML Model Compatibility

    • Risk: TensorFlow models may not load correctly
    • Mitigation: Test model loading early, keep model files unchanged

  2. Database Performance

    • Risk: Async queries may have different performance characteristics
    • Mitigation: Performance test early, optimize queries, add indexes

  3. External API Changes

    • Risk: Maccabi API or other integrations may break
    • Mitigation: Test integrations thoroughly, add error handling

  4. Multi-Tenancy Bugs

    • Risk: Data leakage between tenants
    • Mitigation: Extensive testing, code reviews, security audit

Medium-Risk Items

  1. Authentication Edge Cases

    • Risk: Some auth flows may not work correctly
    • Mitigation: Test all authentication methods, add comprehensive tests

  2. Cloud Storage Migration

    • Risk: Image paths or storage logic may break
    • Mitigation: Test with real storage accounts, add fallback logic

  3. Translation Accuracy

    • Risk: Translations may be incorrect or missing
    • Mitigation: Review translation files, test with native speakers

Timeline Summary

PhaseDurationKey Deliverables
Phase 1: FoundationWeek 1Monorepo structure, py_core, database schema
Phase 2: AI ServiceWeeks 2-4Fully functional AI service with all endpoints
Phase 3: API GatewayWeeks 5-8Fully functional API gateway with all endpoints
Phase 4: IntegrationWeeks 9-10All tests passing, performance validated
Phase 5: DeploymentWeeks 11-12Production-ready, documented, deployed

Total Duration: 12 weeks

Post-Migration

Monitoring & Maintenance

  • Set up application monitoring (Prometheus + Grafana)
  • Set up error tracking (Sentry)
  • Set up log aggregation (ELK or similar)
  • Create on-call runbooks
  • Schedule regular security audits
  • Plan for dependency updates

Future Enhancements

  • WebSocket support for real-time updates
  • GraphQL API as alternative
  • Batch processing with Celery
  • ML pipeline automation
  • Data warehouse integration
  • Mobile SDKs
Last updated on
GPU SetupDevelopment