Skip to main content
EVOKORE// BROWSE
>

./browse/prompts

17 NODES
πŸ€–system promptβ€’6 months ago

Hook Observability and Session Replay

Instrument hooks with replayable logs, task-state visibility, and non-blocking observability around agent sessions.

devops
⭐1
# Hook Observability and Session Replay Imported from curated first-party documentation sources. ## What this covers Use this skill when you need to understand what hooks fired, what they emitted, and how a session can be replayed after the fact. ## Use this when - Debugging hook-driven automation - Replaying session events after failures - Adding observability without blocking interactive work ## Expected outcomes - Hook activity becomes inspectable and replayable - Operational state survives beyond a single terminal session - Observability stays useful without overwhelming operators ## Source synthesis - EVOKORE-MCP/docs/VOICE_AND_HOOKS.md (https://github.com/mattmre/EVOKORE-MCP/blob/main/docs/VOICE_AND_HOOKS.md) - EVOKORE-MCP/docs/USE_CASES_AND_WALKTHROUGHS.md (https://github.com/mattmre/EVOKORE-MCP/blob/main/docs/USE_CASES_AND_WALKTHROUGHS.md) ## Dedupe notes Focuses on replay and observability instead of importing the broader voice-sidecar guide verbatim. ## Source excerpts ### EVOKORE-MCP/docs/VOICE_AND_HOOKS.md EVOKORE currently has three separate voice-related systems plus a set of hook and observability utilities. They overlap in operator workflows, but they are not the same runtime. ## The three voice-related systems ### 1. ElevenLabs MCP proxy This is the optional `elevenlabs` child server configured in `mcp.config.json`. What it is: - proxied through the EVOKORE router - exposed as prefixed MCP tools - available to any EVOKORE-connected MCP client when configured successfully What it is for: - text-to-speech and other ElevenLabs MCP operations as tools - routing voice-related actions through the standard EVOKORE proxy/security stack Requirements: - `uvx` available on PATH - `ELEVENLABS_API_KEY` set ### 2. VoiceMode VoiceMode is a separate voice-conversation system for Claude Code. What it is: - registered separately from EVOKORE - not routed through EVOKOREÒ€ℒs stdio server - used for bidirectional voice conversation in Claude Code What it is for: - speaking to Claude and hearing spoken responses - using `OPENAI_API_KEY` and VoiceModeÒ€ℒs own runtime Windows note: - VoiceMode relies on `uvx` being directly available - set `OPENAI_API_KEY` in the shell that launches Claude Code ### 3. VoiceSidecar VoiceSidecar is a standalone WebSocket server implemented in `src/Voice ... ### EVOKORE-MCP/docs/USE_CASES_AND_WALKTHROUGHS.md This guide turns the runtime contracts into practical operator flows. ## Walkthrough 1: Adopt a workflow from the skill library Use this when you want EVOKORE to retrieve process guidance before the model starts acting. ### Goal Find and adopt an existing workflow such as `session-wrap`. ### Steps 1. Ask the client to search skills: ```text Search the MCP for a workflow about session wrap-up and continuity. ``` 2. EVOKORE uses `search_skills` and returns matching skills. 3. Ask for a specific skill: ```text Show me help for the session-wrap skill. ``` 4. EVOKORE uses `get_skill_help` and returns the skillÒ€ℒs internal instructions. 5. For broader task matching, ask: ```text Resolve a workflow for wrapping this session, documenting open risks, and preparing the next handoff. ``` 6. EVOKORE uses `resolve_workflow` and injects the top 1-3 relevant workflows directly into the tool response. ### Why this matters - keeps the model grounded in repo-specific process - reduces prompt drift - makes handoff and governance behavior repeatable ## Walkthrough 2: Use a proxied tool that requires HITL approval Use this when the tool is configured as `require_approval` in `permissions.yml`. ### Goal Allow a protected proxied tool call such as `fs_write_f ...
πŸ‘0
πŸ‘οΈ0
docs
πŸ€–system promptβ€’7 months ago

team-composition-patterns

Design optimal agent team compositions with sizing heuristics,

coding
⭐1
# Team Composition Patterns Best practices for composing multi-agent teams, selecting team sizes, choosing agent types, and configuring display modes for Claude Code's Agent Teams feature. ## When to Use This Skill - Deciding how many teammates to spawn for a task - Choosing between preset team configurations - Selecting the right agent type (subagent_type) for each role - Configuring teammate display modes (tmux, iTerm2, in-process) - Building custom team compositions for non-standard workflows ## Team Sizing Heuristics | Complexity | Team Size | When to Use | | ------------ | --------- | ----------------------------------------------------------- | | Simple | 1-2 | Single-dimension review, isolated bug, small feature | | Moderate | 2-3 | Multi-file changes, 2-3 concerns, medium features | | Complex | 3-4 | Cross-cutting concerns, large features, deep debugging | | Very Complex | 4-5 | Full-stack features, comprehensive reviews, systemic issues | **Rule of thumb**: Start with the smallest team that covers all required dimensions. Adding teammates increases coordination overhead. ## Preset Team Compositions ### Review Team - **Size**: 3 reviewers - **Agents**: 3x `team-reviewer` - **Default dimensions**: security, performance, architecture - **Use when**: Code changes need multi-dimensional quality assessment ### Debug Team - **Size**: 3 investigators - **Agents**: 3x `team-debugger` - **Default hypotheses**: 3 competing hypotheses - **Use when**: Bug has multiple plausible root causes ### Feature Team - **Size**: 3 (1 lead + 2 implementers) - **Agents**: 1x `team-lead` + 2x `team-implementer` - **Use when**: Feature can be decomposed into parallel work streams ### Fullstack Team - **Size**: 4 (1 lead + 3 implementers) - **Agents**: 1x `team-lead` + 1x frontend `team-implementer` + 1x backend `team-implementer` + 1x test `team-implementer` - **Use when**: Feature spans frontend, backend, and test layers ### Research Team - **Size**: 3 researchers - **Agents**: 3x `general-purpose` - **Default areas**: Each assigned a different research question, module, or topic - **Capabilities**: Codebase search (Grep, Glob, Read), web search (WebSearch, WebFetch) - **Use when**: Need to understand a codebase, research libraries, compare approaches, or gather information from code and web sources in parallel ### Security Team - **Size**: 4 reviewers - **Agents**: 4x `team-reviewer` - **Default dimensions**: OWASP/vulnerabilities, auth/access control, dependencies/supply chain, secrets/configuration - **Use when**: Comprehensive security audit covering multiple attack surfaces ### Migration Team - **Size**: 4 (1 lead + 2 implementers + 1 reviewer) - **Agents**: 1x `team-lead` + 2x `team-implementer` + 1x `team-reviewer` - **Use when**: Large codebase migration (framework upgrade, language port, API version bump) requiring parallel work with correctness verification ## Agent Type Selection When spawning teammates with the Task tool, choose `subagent_type` based on what tools the teammate needs: | Agent Type | Tools Available | Use For | | ------------------------------ | ----------------------------------------- | ---------------------------------------------------------- | | `general-purpose` | All tools (Read, Write, Edit, Bash, etc.) | Implementation, debugging, any task requiring file changes | | `Explore` | Read-only tools (Read, Grep, Glob) | Research, code exploration, analysis | | `Plan` | Read-only tools | Architecture planning, task decomposition | | `agent-teams:team-reviewer` | All tools | Code review with structured findings | | `agent-teams:team-debugger` | All tools | Hypothesis-driven investigation | | `agent-teams:team-implementer` | All tools | Building features within file ownership boundaries | | `agent-teams:team-lead` | All tools | Team orchestration and coordination | **Key distinction**: Read-only agents (Explore, Plan) cannot modify files. Never assign implementation tasks to read-only agents. ## Display Mode Configuration Configure in `~/.claude/settings.json`: ```json { "teammateMode": "tmux" } ``` | Mode | Behavior | Best For | | -------------- | ------------------------------ | ------------------------------------------------- | | `"tmux"` | Each teammate in a tmux pane | Development workflows, monitoring multiple agents | | `"iterm2"` | Each teammate in an iTerm2 tab | macOS users who prefer iTerm2 | | `"in-process"` | All teammates in same process | Simple tasks, CI/CD environments | ## Custom Team Guidelines When building custom teams: 1. **Every team needs a coordinator** β€” Either designate a `team-lead` or have the user coordinate directly 2. **Match roles to agent types** β€” Use specialized agents (reviewer, debugger, implementer) when available 3. **Avoid duplicate roles** β€” Two agents doing the same thing wastes resources 4. **Define boundaries upfront** β€” Each teammate needs clear ownership of files or responsibilities 5. **Keep it small** β€” 2-4 teammates is the sweet spot; 5+ requires significant coordination overhead
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

architecture-patterns

Implement proven backend architecture patterns including Clean

coding
⭐1
# Architecture Patterns Master proven backend architecture patterns including Clean Architecture, Hexagonal Architecture, and Domain-Driven Design to build maintainable, testable, and scalable systems. ## When to Use This Skill - Designing new backend systems from scratch - Refactoring monolithic applications for better maintainability - Establishing architecture standards for your team - Migrating from tightly coupled to loosely coupled architectures - Implementing domain-driven design principles - Creating testable and mockable codebases - Planning microservices decomposition ## Core Concepts ### 1. Clean Architecture (Uncle Bob) **Layers (dependency flows inward):** - **Entities**: Core business models - **Use Cases**: Application business rules - **Interface Adapters**: Controllers, presenters, gateways - **Frameworks & Drivers**: UI, database, external services **Key Principles:** - Dependencies point inward - Inner layers know nothing about outer layers - Business logic independent of frameworks - Testable without UI, database, or external services ### 2. Hexagonal Architecture (Ports and Adapters) **Components:** - **Domain Core**: Business logic - **Ports**: Interfaces defining interactions - **Adapters**: Implementations of ports (database, REST, message queue) **Benefits:** - Swap implementations easily (mock for testing) - Technology-agnostic core - Clear separation of concerns ### 3. Domain-Driven Design (DDD) **Strategic Patterns:** - **Bounded Contexts**: Separate models for different domains - **Context Mapping**: How contexts relate - **Ubiquitous Language**: Shared terminology **Tactical Patterns:** - **Entities**: Objects with identity - **Value Objects**: Immutable objects defined by attributes - **Aggregates**: Consistency boundaries - **Repositories**: Data access abstraction - **Domain Events**: Things that happened ## Clean Architecture Pattern ### Directory Structure ``` app/ β”œβ”€β”€ domain/ # Entities & business rules β”‚ β”œβ”€β”€ entities/ β”‚ β”‚ β”œβ”€β”€ user.py β”‚ β”‚ └── order.py β”‚ β”œβ”€β”€ value_objects/ β”‚ β”‚ β”œβ”€β”€ email.py β”‚ β”‚ └── money.py β”‚ └── interfaces/ # Abstract interfaces β”‚ β”œβ”€β”€ user_repository.py β”‚ └── payment_gateway.py β”œβ”€β”€ use_cases/ # Application business rules β”‚ β”œβ”€β”€ create_user.py β”‚ β”œβ”€β”€ process_order.py β”‚ └── send_notification.py β”œβ”€β”€ adapters/ # Interface implementations β”‚ β”œβ”€β”€ repositories/ β”‚ β”‚ β”œβ”€β”€ postgres_user_repository.py β”‚ β”‚ └── redis_cache_repository.py β”‚ β”œβ”€β”€ controllers/ β”‚ β”‚ └── user_controller.py β”‚ └── gateways/ β”‚ β”œβ”€β”€ stripe_payment_gateway.py β”‚ └── sendgrid_email_gateway.py └── infrastructure/ # Framework & external concerns β”œβ”€β”€ database.py β”œβ”€β”€ config.py └── logging.py ``` ### Implementation Example ```python # domain/entities/user.py from dataclasses import dataclass from datetime import datetime from typing import Optional @dataclass class User: """Core user entity - no framework dependencies.""" id: str email: str name: str created_at: datetime is_active: bool = True def deactivate(self): """Business rule: deactivating user.""" self.is_active = False def can_place_order(self) -> bool: """Business rule: active users can order.""" return self.is_active # domain/interfaces/user_repository.py from abc import ABC, abstractmethod from typing import Optional, List from domain.entities.user import User class IUserRepository(ABC): """Port: defines contract, no implementation.""" @abstractmethod async def find_by_id(self, user_id: str) -> Optional[User]: pass @abstractmethod async def find_by_email(self, email: str) -> Optional[User]: pass @abstractmethod async def save(self, user: User) -> User: pass @abstractmethod async def delete(self, user_id: str) -> bool: pass # use_cases/create_user.py from domain.entities.user import User from domain.interfaces.user_repository import IUserRepository from dataclasses import dataclass from datetime import datetime import uuid @dataclass class CreateUserRequest: email: str name: str @dataclass class CreateUserResponse: user: User success: bool error: Optional[str] = None class CreateUserUseCase: """Use case: orchestrates business logic.""" def __init__(self, user_repository: IUserRepository): self.user_repository = user_repository async def execute(self, request: CreateUserRequest) -> CreateUserResponse: # Business validation existing = await self.user_repository.find_by_email(request.email) if existing: return CreateUserResponse( user=None, success=False, error="Email already exists" ) # Create entity user = User( id=str(uuid.uuid4()), email=request.email, name=request.name, created_at=datetime.now(), is_active=True ) # Persist saved_user = await self.user_repository.save(user) return CreateUserResponse( user=saved_user, success=True ) # adapters/repositories/postgres_user_repository.py from domain.interfaces.user_repository import IUserRepository from domain.entities.user import User from typing import Optional import asyncpg class PostgresUserRepository(IUserRepository): """Adapter: PostgreSQL implementation.""" def __init__(self, pool: asyncpg.Pool): self.pool = pool async def find_by_id(self, user_id: str) -> Optional[User]: async with self.pool.acquire() as conn: row = await conn.fetchrow( "SELECT * FROM users WHERE id = $1", user_id ) return self._to_entity(row) if row else None async def find_by_email(self, email: str) -> Optional[User]: async with self.pool.acquire() as conn: row = await conn.fetchrow( "SELECT * FROM users WHERE email = $1", email ) return self._to_entity(row) if row else None async def save(self, user: User) -> User: async with self.pool.acquire() as conn: await conn.execute( """ INSERT INTO users (id, email, name, created_at, is_active) VALUES ($1, $2, $3, $4, $5) ON CONFLICT (id) DO UPDATE SET email = $2, name = $3, is_active = $5 """, user.id, user.email, user.name, user.created_at, user.is_active ) return user async def delete(self, user_id: str) -> bool: async with self.pool.acquire() as conn: result = await conn.execute( "DELETE FROM users WHERE id = $1", user_id ) return result == "DELETE 1" def _to_entity(self, row) -> User: """Map database row to entity.""" return User( id=row["id"], email=row["email"], name=row["name"], created_at=row["created_at"], is_active=row["is_active"] ) # adapters/controllers/user_controller.py from fastapi import APIRouter, Depends, HTTPException from use_cases.create_user import CreateUserUseCase, CreateUserRequest from pydantic import BaseModel router = APIRouter() class CreateUserDTO(BaseModel): email: str name: str @router.post("/users") async def create_user( dto: CreateUserDTO, use_case: CreateUserUseCase = Depends(get_create_user_use_case) ): """Controller: handles HTTP concerns only.""" request = CreateUserRequest(email=dto.email, name=dto.name) response = await use_case.execute(request) if not response.success: raise HTTPException(status_code=400, detail=response.error) return {"user": response.user} ``` ## Hexagonal Architecture Pattern ```python # Core domain (hexagon center) class OrderService: """Domain service - no infrastructure dependencies.""" def __init__( self, order_repository: OrderRepositoryPort, payment_gateway: PaymentGatewayPort, notification_service: NotificationPort ): self.orders = order_repository self.payments = payment_gateway self.notifications = notification_service async def place_order(self, order: Order) -> OrderResult: # Business logic if not order.is_valid(): return OrderResult(success=False, error="Invalid order") # Use ports (interfaces) payment = await self.payments.charge( amount=order.total, customer=order.customer_id ) if not payment.success: return OrderResult(success=False, error="Payment failed") order.mark_as_paid() saved_order = await self.orders.save(order) await self.notifications.send( to=order.customer_email, subject="Order confirmed", body=f"Order {order.id} confirmed" ) return OrderResult(success=True, order=saved_order) # Ports (interfaces) class OrderRepositoryPort(ABC): @abstractmethod async def save(self, order: Order) -> Order: pass class PaymentGatewayPort(ABC): @abstractmethod async def charge(self, amount: Money, customer: str) -> PaymentResult: pass class NotificationPort(ABC): @abstractmethod async def send(self, to: str, subject: str, body: str): pass # Adapters (implementations) class StripePaymentAdapter(PaymentGatewayPort): """Primary adapter: connects to Stripe API.""" def __init__(self, api_key: str): self.stripe = stripe self.stripe.api_key = api_key async def charge(self, amount: Money, customer: str) -> PaymentResult: try: charge = self.stripe.Charge.create( amount=amount.cents, currency=amount.currency, customer=customer ) return PaymentResult(success=True, transaction_id=charge.id) except stripe.error.CardError as e: return PaymentResult(success=False, error=str(e)) class MockPaymentAdapter(PaymentGatewayPort): """Test adapter: no external dependencies.""" async def charge(self, amount: Money, customer: str) -> PaymentResult: return PaymentResult(success=True, transaction_id="mock-123") ``` ## Domain-Driven Design Pattern ```python # Value Objects (immutable) from dataclasses import dataclass from typing import Optional @dataclass(frozen=True) class Email: """Value object: validated email.""" value: str def __post_init__(self): if "@" not in self.value: raise ValueError("Invalid email") @dataclass(frozen=True) class Money: """Value object: amount with currency.""" amount: int # cents currency: str def add(self, other: "Money") -> "Money": if self.currency != other.currency: raise ValueError("Currency mismatch") return Money(self.amount + other.amount, self.currency) # Entities (with identity) class Order: """Entity: has identity, mutable state.""" def __init__(self, id: str, customer: Customer): self.id = id self.customer = customer self.items: List[OrderItem] = [] self.status = OrderStatus.PENDING self._events: List[DomainEvent] = [] def add_item(self, product: Product, quantity: int): """Business logic in entity.""" item = OrderItem(product, quantity) self.items.append(item) self._events.append(ItemAddedEvent(self.id, item)) def total(self) -> Money: """Calculated property.""" return sum(item.subtotal() for item in self.items) def submit(self): """State transition with business rules.""" if not self.items: raise ValueError("Cannot submit empty order") if self.status != OrderStatus.PENDING: raise ValueError("Order already submitted") self.status = OrderStatus.SUBMITTED self._events.append(OrderSubmittedEvent(self.id)) # Aggregates (consistency boundary) class Customer: """Aggregate root: controls access to entities.""" def __init__(self, id: str, email: Email): self.id = id self.email = email self._addresses: List[Address] = [] self._orders: List[str] = [] # Order IDs, not full objects def add_address(self, address: Address): """Aggregate enforces invariants.""" if len(self._addresses) >= 5: raise ValueError("Maximum 5 addresses allowed") self._addresses.append(address) @property def primary_address(self) -> Optional[Address]: return next((a for a in self._addresses if a.is_primary), None) # Domain Events @dataclass class OrderSubmittedEvent: order_id: str occurred_at: datetime = field(default_factory=datetime.now) # Repository (aggregate persistence) class OrderRepository: """Repository: persist/retrieve aggregates.""" async def find_by_id(self, order_id: str) -> Optional[Order]: """Reconstitute aggregate from storage.""" pass async def save(self, order: Order): """Persist aggregate and publish events.""" await self._persist(order) await self._publish_events(order._events) order._events.clear() ``` ## Resources - **references/clean-architecture-guide.md**: Detailed layer breakdown - **references/hexagonal-architecture-guide.md**: Ports and adapters patterns - **references/ddd-tactical-patterns.md**: Entities, value objects, aggregates - **assets/clean-architecture-template/**: Complete project structure - **assets/ddd-examples/**: Domain modeling examples ## Best Practices 1. **Dependency Rule**: Dependencies always point inward 2. **Interface Segregation**: Small, focused interfaces 3. **Business Logic in Domain**: Keep frameworks out of core 4. **Test Independence**: Core testable without infrastructure 5. **Bounded Contexts**: Clear domain boundaries 6. **Ubiquitous Language**: Consistent terminology 7. **Thin Controllers**: Delegate to use cases 8. **Rich Domain Models**: Behavior with data ## Common Pitfalls - **Anemic Domain**: Entities with only data, no behavior - **Framework Coupling**: Business logic depends on frameworks - **Fat Controllers**: Business logic in controllers - **Repository Leakage**: Exposing ORM objects - **Missing Abstractions**: Concrete dependencies in core - **Over-Engineering**: Clean architecture for simple CRUD
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

microservices-patterns

Design microservices architectures with service boundaries,

coding
⭐1
# Microservices Patterns Master microservices architecture patterns including service boundaries, inter-service communication, data management, and resilience patterns for building distributed systems. ## When to Use This Skill - Decomposing monoliths into microservices - Designing service boundaries and contracts - Implementing inter-service communication - Managing distributed data and transactions - Building resilient distributed systems - Implementing service discovery and load balancing - Designing event-driven architectures ## Core Concepts ### 1. Service Decomposition Strategies **By Business Capability** - Organize services around business functions - Each service owns its domain - Example: OrderService, PaymentService, InventoryService **By Subdomain (DDD)** - Core domain, supporting subdomains - Bounded contexts map to services - Clear ownership and responsibility **Strangler Fig Pattern** - Gradually extract from monolith - New functionality as microservices - Proxy routes to old/new systems ### 2. Communication Patterns **Synchronous (Request/Response)** - REST APIs - gRPC - GraphQL **Asynchronous (Events/Messages)** - Event streaming (Kafka) - Message queues (RabbitMQ, SQS) - Pub/Sub patterns ### 3. Data Management **Database Per Service** - Each service owns its data - No shared databases - Loose coupling **Saga Pattern** - Distributed transactions - Compensating actions - Eventual consistency ### 4. Resilience Patterns **Circuit Breaker** - Fail fast on repeated errors - Prevent cascade failures **Retry with Backoff** - Transient fault handling - Exponential backoff **Bulkhead** - Isolate resources - Limit impact of failures ## Service Decomposition Patterns ### Pattern 1: By Business Capability ```python # E-commerce example # Order Service class OrderService: """Handles order lifecycle.""" async def create_order(self, order_data: dict) -> Order: order = Order.create(order_data) # Publish event for other services await self.event_bus.publish( OrderCreatedEvent( order_id=order.id, customer_id=order.customer_id, items=order.items, total=order.total ) ) return order # Payment Service (separate service) class PaymentService: """Handles payment processing.""" async def process_payment(self, payment_request: PaymentRequest) -> PaymentResult: # Process payment result = await self.payment_gateway.charge( amount=payment_request.amount, customer=payment_request.customer_id ) if result.success: await self.event_bus.publish( PaymentCompletedEvent( order_id=payment_request.order_id, transaction_id=result.transaction_id ) ) return result # Inventory Service (separate service) class InventoryService: """Handles inventory management.""" async def reserve_items(self, order_id: str, items: List[OrderItem]) -> ReservationResult: # Check availability for item in items: available = await self.inventory_repo.get_available(item.product_id) if available < item.quantity: return ReservationResult( success=False, error=f"Insufficient inventory for {item.product_id}" ) # Reserve items reservation = await self.create_reservation(order_id, items) await self.event_bus.publish( InventoryReservedEvent( order_id=order_id, reservation_id=reservation.id ) ) return ReservationResult(success=True, reservation=reservation) ``` ### Pattern 2: API Gateway ```python from fastapi import FastAPI, HTTPException, Depends import httpx from circuitbreaker import circuit app = FastAPI() class APIGateway: """Central entry point for all client requests.""" def __init__(self): self.order_service_url = "http://order-service:8000" self.payment_service_url = "http://payment-service:8001" self.inventory_service_url = "http://inventory-service:8002" self.http_client = httpx.AsyncClient(timeout=5.0) @circuit(failure_threshold=5, recovery_timeout=30) async def call_order_service(self, path: str, method: str = "GET", **kwargs): """Call order service with circuit breaker.""" response = await self.http_client.request( method, f"{self.order_service_url}{path}", **kwargs ) response.raise_for_status() return response.json() async def create_order_aggregate(self, order_id: str) -> dict: """Aggregate data from multiple services.""" # Parallel requests order, payment, inventory = await asyncio.gather( self.call_order_service(f"/orders/{order_id}"), self.call_payment_service(f"/payments/order/{order_id}"), self.call_inventory_service(f"/reservations/order/{order_id}"), return_exceptions=True ) # Handle partial failures result = {"order": order} if not isinstance(payment, Exception): result["payment"] = payment if not isinstance(inventory, Exception): result["inventory"] = inventory return result @app.post("/api/orders") async def create_order( order_data: dict, gateway: APIGateway = Depends() ): """API Gateway endpoint.""" try: # Route to order service order = await gateway.call_order_service( "/orders", method="POST", json=order_data ) return {"order": order} except httpx.HTTPError as e: raise HTTPException(status_code=503, detail="Order service unavailable") ``` ## Communication Patterns ### Pattern 1: Synchronous REST Communication ```python # Service A calls Service B import httpx from tenacity import retry, stop_after_attempt, wait_exponential class ServiceClient: """HTTP client with retries and timeout.""" def __init__(self, base_url: str): self.base_url = base_url self.client = httpx.AsyncClient( timeout=httpx.Timeout(5.0, connect=2.0), limits=httpx.Limits(max_keepalive_connections=20) ) @retry( stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=2, max=10) ) async def get(self, path: str, **kwargs): """GET with automatic retries.""" response = await self.client.get(f"{self.base_url}{path}", **kwargs) response.raise_for_status() return response.json() async def post(self, path: str, **kwargs): """POST request.""" response = await self.client.post(f"{self.base_url}{path}", **kwargs) response.raise_for_status() return response.json() # Usage payment_client = ServiceClient("http://payment-service:8001") result = await payment_client.post("/payments", json=payment_data) ``` ### Pattern 2: Asynchronous Event-Driven ```python # Event-driven communication with Kafka from aiokafka import AIOKafkaProducer, AIOKafkaConsumer import json from dataclasses import dataclass, asdict from datetime import datetime @dataclass class DomainEvent: event_id: str event_type: str aggregate_id: str occurred_at: datetime data: dict class EventBus: """Event publishing and subscription.""" def __init__(self, bootstrap_servers: List[str]): self.bootstrap_servers = bootstrap_servers self.producer = None async def start(self): self.producer = AIOKafkaProducer( bootstrap_servers=self.bootstrap_servers, value_serializer=lambda v: json.dumps(v).encode() ) await self.producer.start() async def publish(self, event: DomainEvent): """Publish event to Kafka topic.""" topic = event.event_type await self.producer.send_and_wait( topic, value=asdict(event), key=event.aggregate_id.encode() ) async def subscribe(self, topic: str, handler: callable): """Subscribe to events.""" consumer = AIOKafkaConsumer( topic, bootstrap_servers=self.bootstrap_servers, value_deserializer=lambda v: json.loads(v.decode()), group_id="my-service" ) await consumer.start() try: async for message in consumer: event_data = message.value await handler(event_data) finally: await consumer.stop() # Order Service publishes event async def create_order(order_data: dict): order = await save_order(order_data) event = DomainEvent( event_id=str(uuid.uuid4()), event_type="OrderCreated", aggregate_id=order.id, occurred_at=datetime.now(), data={ "order_id": order.id, "customer_id": order.customer_id, "total": order.total } ) await event_bus.publish(event) # Inventory Service listens for OrderCreated async def handle_order_created(event_data: dict): """React to order creation.""" order_id = event_data["data"]["order_id"] items = event_data["data"]["items"] # Reserve inventory await reserve_inventory(order_id, items) ``` ### Pattern 3: Saga Pattern (Distributed Transactions) ```python # Saga orchestration for order fulfillment from enum import Enum from typing import List, Callable class SagaStep: """Single step in saga.""" def __init__( self, name: str, action: Callable, compensation: Callable ): self.name = name self.action = action self.compensation = compensation class SagaStatus(Enum): PENDING = "pending" COMPLETED = "completed" COMPENSATING = "compensating" FAILED = "failed" class OrderFulfillmentSaga: """Orchestrated saga for order fulfillment.""" def __init__(self): self.steps: List[SagaStep] = [ SagaStep( "create_order", action=self.create_order, compensation=self.cancel_order ), SagaStep( "reserve_inventory", action=self.reserve_inventory, compensation=self.release_inventory ), SagaStep( "process_payment", action=self.process_payment, compensation=self.refund_payment ), SagaStep( "confirm_order", action=self.confirm_order, compensation=self.cancel_order_confirmation ) ] async def execute(self, order_data: dict) -> SagaResult: """Execute saga steps.""" completed_steps = [] context = {"order_data": order_data} try: for step in self.steps: # Execute step result = await step.action(context) if not result.success: # Compensate await self.compensate(completed_steps, context) return SagaResult( status=SagaStatus.FAILED, error=result.error ) completed_steps.append(step) context.update(result.data) return SagaResult(status=SagaStatus.COMPLETED, data=context) except Exception as e: # Compensate on error await self.compensate(completed_steps, context) return SagaResult(status=SagaStatus.FAILED, error=str(e)) async def compensate(self, completed_steps: List[SagaStep], context: dict): """Execute compensating actions in reverse order.""" for step in reversed(completed_steps): try: await step.compensation(context) except Exception as e: # Log compensation failure print(f"Compensation failed for {step.name}: {e}") # Step implementations async def create_order(self, context: dict) -> StepResult: order = await order_service.create(context["order_data"]) return StepResult(success=True, data={"order_id": order.id}) async def cancel_order(self, context: dict): await order_service.cancel(context["order_id"]) async def reserve_inventory(self, context: dict) -> StepResult: result = await inventory_service.reserve( context["order_id"], context["order_data"]["items"] ) return StepResult( success=result.success, data={"reservation_id": result.reservation_id} ) async def release_inventory(self, context: dict): await inventory_service.release(context["reservation_id"]) async def process_payment(self, context: dict) -> StepResult: result = await payment_service.charge( context["order_id"], context["order_data"]["total"] ) return StepResult( success=result.success, data={"transaction_id": result.transaction_id}, error=result.error ) async def refund_payment(self, context: dict): await payment_service.refund(context["transaction_id"]) ``` ## Resilience Patterns ### Circuit Breaker Pattern ```python from enum import Enum from datetime import datetime, timedelta from typing import Callable, Any class CircuitState(Enum): CLOSED = "closed" # Normal operation OPEN = "open" # Failing, reject requests HALF_OPEN = "half_open" # Testing if recovered class CircuitBreaker: """Circuit breaker for service calls.""" def __init__( self, failure_threshold: int = 5, recovery_timeout: int = 30, success_threshold: int = 2 ): self.failure_threshold = failure_threshold self.recovery_timeout = recovery_timeout self.success_threshold = success_threshold self.failure_count = 0 self.success_count = 0 self.state = CircuitState.CLOSED self.opened_at = None async def call(self, func: Callable, *args, **kwargs) -> Any: """Execute function with circuit breaker.""" if self.state == CircuitState.OPEN: if self._should_attempt_reset(): self.state = CircuitState.HALF_OPEN else: raise CircuitBreakerOpenError("Circuit breaker is open") try: result = await func(*args, **kwargs) self._on_success() return result except Exception as e: self._on_failure() raise def _on_success(self): """Handle successful call.""" self.failure_count = 0 if self.state == CircuitState.HALF_OPEN: self.success_count += 1 if self.success_count >= self.success_threshold: self.state = CircuitState.CLOSED self.success_count = 0 def _on_failure(self): """Handle failed call.""" self.failure_count += 1 if self.failure_count >= self.failure_threshold: self.state = CircuitState.OPEN self.opened_at = datetime.now() if self.state == CircuitState.HALF_OPEN: self.state = CircuitState.OPEN self.opened_at = datetime.now() def _should_attempt_reset(self) -> bool: """Check if enough time passed to try again.""" return ( datetime.now() - self.opened_at > timedelta(seconds=self.recovery_timeout) ) # Usage breaker = CircuitBreaker(failure_threshold=5, recovery_timeout=30) async def call_payment_service(payment_data: dict): return await breaker.call( payment_client.process_payment, payment_data ) ``` ## Resources - **references/service-decomposition-guide.md**: Breaking down monoliths - **references/communication-patterns.md**: Sync vs async patterns - **references/saga-implementation.md**: Distributed transactions - **assets/circuit-breaker.py**: Production circuit breaker - **assets/event-bus-template.py**: Kafka event bus implementation - **assets/api-gateway-template.py**: Complete API gateway ## Best Practices 1. **Service Boundaries**: Align with business capabilities 2. **Database Per Service**: No shared databases 3. **API Contracts**: Versioned, backward compatible 4. **Async When Possible**: Events over direct calls 5. **Circuit Breakers**: Fail fast on service failures 6. **Distributed Tracing**: Track requests across services 7. **Service Registry**: Dynamic service discovery 8. **Health Checks**: Liveness and readiness probes ## Common Pitfalls - **Distributed Monolith**: Tightly coupled services - **Chatty Services**: Too many inter-service calls - **Shared Databases**: Tight coupling through data - **No Circuit Breakers**: Cascade failures - **Synchronous Everything**: Tight coupling, poor resilience - **Premature Microservices**: Starting with microservices - **Ignoring Network Failures**: Assuming reliable network - **No Compensation Logic**: Can't undo failed transactions
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

data-storytelling

Transform data into compelling narratives using visualization,

data
⭐1
# Data Storytelling Transform raw data into compelling narratives that drive decisions and inspire action. ## When to Use This Skill - Presenting analytics to executives - Creating quarterly business reviews - Building investor presentations - Writing data-driven reports - Communicating insights to non-technical audiences - Making recommendations based on data ## Core Concepts ### 1. Story Structure ``` Setup β†’ Conflict β†’ Resolution Setup: Context and baseline Conflict: The problem or opportunity Resolution: Insights and recommendations ``` ### 2. Narrative Arc ``` 1. Hook: Grab attention with surprising insight 2. Context: Establish the baseline 3. Rising Action: Build through data points 4. Climax: The key insight 5. Resolution: Recommendations 6. Call to Action: Next steps ``` ### 3. Three Pillars | Pillar | Purpose | Components | | ------------- | -------- | -------------------------------- | | **Data** | Evidence | Numbers, trends, comparisons | | **Narrative** | Meaning | Context, causation, implications | | **Visuals** | Clarity | Charts, diagrams, highlights | ## Story Frameworks ### Framework 1: The Problem-Solution Story ```markdown # Customer Churn Analysis ## The Hook "We're losing $2.4M annually to preventable churn." ## The Context - Current churn rate: 8.5% (industry average: 5%) - Average customer lifetime value: $4,800 - 500 customers churned last quarter ## The Problem Analysis of churned customers reveals a pattern: - 73% churned within first 90 days - Common factor: < 3 support interactions - Low feature adoption in first month ## The Insight [Show engagement curve visualization] Customers who don't engage in the first 14 days are 4x more likely to churn. ## The Solution 1. Implement 14-day onboarding sequence 2. Proactive outreach at day 7 3. Feature adoption tracking ## Expected Impact - Reduce early churn by 40% - Save $960K annually - Payback period: 3 months ## Call to Action Approve $50K budget for onboarding automation. ``` ### Framework 2: The Trend Story ```markdown # Q4 Performance Analysis ## Where We Started Q3 ended with $1.2M MRR, 15% below target. Team morale was low after missed goals. ## What Changed [Timeline visualization] - Oct: Launched self-serve pricing - Nov: Reduced friction in signup - Dec: Added customer success calls ## The Transformation [Before/after comparison chart] | Metric | Q3 | Q4 | Change | |----------------|--------|--------|--------| | Trial β†’ Paid | 8% | 15% | +87% | | Time to Value | 14 days| 5 days | -64% | | Expansion Rate | 2% | 8% | +300% | ## Key Insight Self-serve + high-touch creates compound growth. Customers who self-serve AND get a success call have 3x higher expansion rate. ## Going Forward Double down on hybrid model. Target: $1.8M MRR by Q2. ``` ### Framework 3: The Comparison Story ```markdown # Market Opportunity Analysis ## The Question Should we expand into EMEA or APAC first? ## The Comparison [Side-by-side market analysis] ### EMEA - Market size: $4.2B - Growth rate: 8% - Competition: High - Regulatory: Complex (GDPR) - Language: Multiple ### APAC - Market size: $3.8B - Growth rate: 15% - Competition: Moderate - Regulatory: Varied - Language: Multiple ## The Analysis [Weighted scoring matrix visualization] | Factor | Weight | EMEA Score | APAC Score | | ----------- | ------ | ---------- | ---------- | | Market Size | 25% | 5 | 4 | | Growth | 30% | 3 | 5 | | Competition | 20% | 2 | 4 | | Ease | 25% | 2 | 3 | | **Total** | | **2.9** | **4.1** | ## The Recommendation APAC first. Higher growth, less competition. Start with Singapore hub (English, business-friendly). Enter EMEA in Year 2 with localization ready. ## Risk Mitigation - Timezone coverage: Hire 24/7 support - Cultural fit: Local partnerships - Payment: Multi-currency from day 1 ``` ## Visualization Techniques ### Technique 1: Progressive Reveal ```markdown Start simple, add layers: Slide 1: "Revenue is growing" [single line chart] Slide 2: "But growth is slowing" [add growth rate overlay] Slide 3: "Driven by one segment" [add segment breakdown] Slide 4: "Which is saturating" [add market share] Slide 5: "We need new segments" [add opportunity zones] ``` ### Technique 2: Contrast and Compare ```markdown Before/After: β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ BEFORE β”‚ AFTER β”‚ β”‚ β”‚ β”‚ β”‚ Process: 5 daysβ”‚ Process: 1 day β”‚ β”‚ Errors: 15% β”‚ Errors: 2% β”‚ β”‚ Cost: $50/unit β”‚ Cost: $20/unit β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ This/That (emphasize difference): β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ CUSTOMER A vs B β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ β”‚ β”‚ β–ˆβ–ˆ β”‚ β”‚ β”‚ β”‚ $45,000 β”‚ β”‚ $8,000 β”‚ β”‚ β”‚ β”‚ LTV β”‚ β”‚ LTV β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ Onboarded No onboarding β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` ### Technique 3: Annotation and Highlight ```python import matplotlib.pyplot as plt import pandas as pd fig, ax = plt.subplots(figsize=(12, 6)) # Plot the main data ax.plot(dates, revenue, linewidth=2, color='#2E86AB') # Add annotation for key events ax.annotate( 'Product Launch\n+32% spike', xy=(launch_date, launch_revenue), xytext=(launch_date, launch_revenue * 1.2), fontsize=10, arrowprops=dict(arrowstyle='->', color='#E63946'), color='#E63946' ) # Highlight a region ax.axvspan(growth_start, growth_end, alpha=0.2, color='green', label='Growth Period') # Add threshold line ax.axhline(y=target, color='gray', linestyle='--', label=f'Target: ${target:,.0f}') ax.set_title('Revenue Growth Story', fontsize=14, fontweight='bold') ax.legend() ``` ## Presentation Templates ### Template 1: Executive Summary Slide ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ KEY INSIGHT β”‚ β”‚ ══════════════════════════════════════════════════════════│ β”‚ β”‚ β”‚ "Customers who complete onboarding in week 1 β”‚ β”‚ have 3x higher lifetime value" β”‚ β”‚ β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ β”‚ β”‚ β”‚ THE DATA β”‚ THE IMPLICATION β”‚ β”‚ β”‚ β”‚ β”‚ Week 1 completers: β”‚ βœ“ Prioritize onboarding UX β”‚ β”‚ β€’ LTV: $4,500 β”‚ βœ“ Add day-1 success milestones β”‚ β”‚ β€’ Retention: 85% β”‚ βœ“ Proactive week-1 outreach β”‚ β”‚ β€’ NPS: 72 β”‚ β”‚ β”‚ β”‚ Investment: $75K β”‚ β”‚ Others: β”‚ Expected ROI: 8x β”‚ β”‚ β€’ LTV: $1,500 β”‚ β”‚ β”‚ β€’ Retention: 45% β”‚ β”‚ β”‚ β€’ NPS: 34 β”‚ β”‚ β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` ### Template 2: Data Story Flow ``` Slide 1: THE HEADLINE "We can grow 40% faster by fixing onboarding" Slide 2: THE CONTEXT Current state metrics Industry benchmarks Gap analysis Slide 3: THE DISCOVERY What the data revealed Surprising finding Pattern identification Slide 4: THE DEEP DIVE Root cause analysis Segment breakdowns Statistical significance Slide 5: THE RECOMMENDATION Proposed actions Resource requirements Timeline Slide 6: THE IMPACT Expected outcomes ROI calculation Risk assessment Slide 7: THE ASK Specific request Decision needed Next steps ``` ### Template 3: One-Page Dashboard Story ```markdown # Monthly Business Review: January 2024 ## THE HEADLINE Revenue up 15% but CAC increasing faster than LTV ## KEY METRICS AT A GLANCE β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ MRR β”‚ NRR β”‚ CAC β”‚ LTV β”‚ β”‚ $125K β”‚ 108% β”‚ $450 β”‚ $2,200 β”‚ β”‚ β–²15% β”‚ β–²3% β”‚ β–²22% β”‚ β–²8% β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ## WHAT'S WORKING βœ“ Enterprise segment growing 25% MoM βœ“ Referral program driving 30% of new logos βœ“ Support satisfaction at all-time high (94%) ## WHAT NEEDS ATTENTION βœ— SMB acquisition cost up 40% βœ— Trial conversion down 5 points βœ— Time-to-value increased by 3 days ## ROOT CAUSE [Mini chart showing SMB vs Enterprise CAC trend] SMB paid ads becoming less efficient. CPC up 35% while conversion flat. ## RECOMMENDATION 1. Shift $20K/mo from paid to content 2. Launch SMB self-serve trial 3. A/B test shorter onboarding ## NEXT MONTH'S FOCUS - Launch content marketing pilot - Complete self-serve MVP - Reduce time-to-value to < 7 days ``` ## Writing Techniques ### Headlines That Work ```markdown BAD: "Q4 Sales Analysis" GOOD: "Q4 Sales Beat Target by 23% - Here's Why" BAD: "Customer Churn Report" GOOD: "We're Losing $2.4M to Preventable Churn" BAD: "Marketing Performance" GOOD: "Content Marketing Delivers 4x ROI vs. Paid" Formula: [Specific Number] + [Business Impact] + [Actionable Context] ``` ### Transition Phrases ```markdown Building the narrative: β€’ "This leads us to ask..." β€’ "When we dig deeper..." β€’ "The pattern becomes clear when..." β€’ "Contrast this with..." Introducing insights: β€’ "The data reveals..." β€’ "What surprised us was..." β€’ "The inflection point came when..." β€’ "The key finding is..." Moving to action: β€’ "This insight suggests..." β€’ "Based on this analysis..." β€’ "The implication is clear..." β€’ "Our recommendation is..." ``` ### Handling Uncertainty ```markdown Acknowledge limitations: β€’ "With 95% confidence, we can say..." β€’ "The sample size of 500 shows..." β€’ "While correlation is strong, causation requires..." β€’ "This trend holds for [segment], though [caveat]..." Present ranges: β€’ "Impact estimate: $400K-$600K" β€’ "Confidence interval: 15-20% improvement" β€’ "Best case: X, Conservative: Y" ``` ## Best Practices ### Do's - **Start with the "so what"** - Lead with insight - **Use the rule of three** - Three points, three comparisons - **Show, don't tell** - Let data speak - **Make it personal** - Connect to audience goals - **End with action** - Clear next steps ### Don'ts - **Don't data dump** - Curate ruthlessly - **Don't bury the insight** - Front-load key findings - **Don't use jargon** - Match audience vocabulary - **Don't show methodology first** - Context, then method - **Don't forget the narrative** - Numbers need meaning ## Resources - [Storytelling with Data (Cole Nussbaumer)](https://www.storytellingwithdata.com/) - [The Pyramid Principle (Barbara Minto)](https://www.amazon.com/Pyramid-Principle-Logic-Writing-Thinking/dp/0273710516) - [Resonate (Nancy Duarte)](https://www.duarte.com/resonate/)
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

cost-optimization

Optimize cloud costs through resource rightsizing, tagging

architecture
⭐1
# Cloud Cost Optimization Strategies and patterns for optimizing cloud costs across AWS, Azure, and GCP. ## Purpose Implement systematic cost optimization strategies to reduce cloud spending while maintaining performance and reliability. ## When to Use - Reduce cloud spending - Right-size resources - Implement cost governance - Optimize multi-cloud costs - Meet budget constraints ## Cost Optimization Framework ### 1. Visibility - Implement cost allocation tags - Use cloud cost management tools - Set up budget alerts - Create cost dashboards ### 2. Right-Sizing - Analyze resource utilization - Downsize over-provisioned resources - Use auto-scaling - Remove idle resources ### 3. Pricing Models - Use reserved capacity - Leverage spot/preemptible instances - Implement savings plans - Use committed use discounts ### 4. Architecture Optimization - Use managed services - Implement caching - Optimize data transfer - Use lifecycle policies ## AWS Cost Optimization ### Reserved Instances ``` Savings: 30-72% vs On-Demand Term: 1 or 3 years Payment: All/Partial/No upfront Flexibility: Standard or Convertible ``` ### Savings Plans ``` Compute Savings Plans: 66% savings EC2 Instance Savings Plans: 72% savings Applies to: EC2, Fargate, Lambda Flexible across: Instance families, regions, OS ``` ### Spot Instances ``` Savings: Up to 90% vs On-Demand Best for: Batch jobs, CI/CD, stateless workloads Risk: 2-minute interruption notice Strategy: Mix with On-Demand for resilience ``` ### S3 Cost Optimization ```hcl resource "aws_s3_bucket_lifecycle_configuration" "example" { bucket = aws_s3_bucket.example.id rule { id = "transition-to-ia" status = "Enabled" transition { days = 30 storage_class = "STANDARD_IA" } transition { days = 90 storage_class = "GLACIER" } expiration { days = 365 } } } ``` ## Azure Cost Optimization ### Reserved VM Instances - 1 or 3 year terms - Up to 72% savings - Flexible sizing - Exchangeable ### Azure Hybrid Benefit - Use existing Windows Server licenses - Up to 80% savings with RI - Available for Windows and SQL Server ### Azure Advisor Recommendations - Right-size VMs - Delete unused resources - Use reserved capacity - Optimize storage ## GCP Cost Optimization ### Committed Use Discounts - 1 or 3 year commitment - Up to 57% savings - Applies to vCPUs and memory - Resource-based or spend-based ### Sustained Use Discounts - Automatic discounts - Up to 30% for running instances - No commitment required - Applies to Compute Engine, GKE ### Preemptible VMs - Up to 80% savings - 24-hour maximum runtime - Best for batch workloads ## Tagging Strategy ### AWS Tagging ```hcl locals { common_tags = { Environment = "production" Project = "my-project" CostCenter = "engineering" Owner = "team@example.com" ManagedBy = "terraform" } } resource "aws_instance" "example" { ami = "ami-12345678" instance_type = "t3.medium" tags = merge( local.common_tags, { Name = "web-server" } ) } ``` **Reference:** See `references/tagging-standards.md` ## Cost Monitoring ### Budget Alerts ```hcl # AWS Budget resource "aws_budgets_budget" "monthly" { name = "monthly-budget" budget_type = "COST" limit_amount = "1000" limit_unit = "USD" time_period_start = "2024-01-01_00:00" time_unit = "MONTHLY" notification { comparison_operator = "GREATER_THAN" threshold = 80 threshold_type = "PERCENTAGE" notification_type = "ACTUAL" subscriber_email_addresses = ["team@example.com"] } } ``` ### Cost Anomaly Detection - AWS Cost Anomaly Detection - Azure Cost Management alerts - GCP Budget alerts ## Architecture Patterns ### Pattern 1: Serverless First - Use Lambda/Functions for event-driven - Pay only for execution time - Auto-scaling included - No idle costs ### Pattern 2: Right-Sized Databases ``` Development: t3.small RDS Staging: t3.large RDS Production: r6g.2xlarge RDS with read replicas ``` ### Pattern 3: Multi-Tier Storage ``` Hot data: S3 Standard Warm data: S3 Standard-IA (30 days) Cold data: S3 Glacier (90 days) Archive: S3 Deep Archive (365 days) ``` ### Pattern 4: Auto-Scaling ```hcl resource "aws_autoscaling_policy" "scale_up" { name = "scale-up" scaling_adjustment = 2 adjustment_type = "ChangeInCapacity" cooldown = 300 autoscaling_group_name = aws_autoscaling_group.main.name } resource "aws_cloudwatch_metric_alarm" "cpu_high" { alarm_name = "cpu-high" comparison_operator = "GreaterThanThreshold" evaluation_periods = "2" metric_name = "CPUUtilization" namespace = "AWS/EC2" period = "60" statistic = "Average" threshold = "80" alarm_actions = [aws_autoscaling_policy.scale_up.arn] } ``` ## Cost Optimization Checklist - [ ] Implement cost allocation tags - [ ] Delete unused resources (EBS, EIPs, snapshots) - [ ] Right-size instances based on utilization - [ ] Use reserved capacity for steady workloads - [ ] Implement auto-scaling - [ ] Optimize storage classes - [ ] Use lifecycle policies - [ ] Enable cost anomaly detection - [ ] Set budget alerts - [ ] Review costs weekly - [ ] Use spot/preemptible instances - [ ] Optimize data transfer costs - [ ] Implement caching layers - [ ] Use managed services - [ ] Monitor and optimize continuously ## Tools - **AWS:** Cost Explorer, Cost Anomaly Detection, Compute Optimizer - **Azure:** Cost Management, Advisor - **GCP:** Cost Management, Recommender - **Multi-cloud:** CloudHealth, Cloudability, Kubecost ## Reference Files - `references/tagging-standards.md` - Tagging conventions - `assets/cost-analysis-template.xlsx` - Cost analysis spreadsheet ## Related Skills - `terraform-module-library` - For resource provisioning - `multi-cloud-architecture` - For cloud selection
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

context-driven-development

Creates and maintains project context artifacts (product.md,

coding
⭐1
# Context-Driven Development Guide for implementing and maintaining context as a managed artifact alongside code, enabling consistent AI interactions and team alignment through structured project documentation. ## When to Use This Skill - Setting up new projects with Conductor - Understanding the relationship between context artifacts - Maintaining consistency across AI-assisted development sessions - Onboarding team members to an existing Conductor project - Deciding when to update context documents - Managing greenfield vs brownfield project contexts ## Core Philosophy Context-Driven Development treats project context as a first-class artifact managed alongside code. Instead of relying on ad-hoc prompts or scattered documentation, establish a persistent, structured foundation that informs all AI interactions. Key principles: 1. **Context precedes code**: Define what you're building and how before implementation 2. **Living documentation**: Context artifacts evolve with the project 3. **Single source of truth**: One canonical location for each type of information 4. **AI alignment**: Consistent context produces consistent AI behavior ## The Workflow Follow the **Context β†’ Spec & Plan β†’ Implement** workflow: 1. **Context Phase**: Establish or verify project context artifacts exist and are current 2. **Specification Phase**: Define requirements and acceptance criteria for work units 3. **Planning Phase**: Break specifications into phased, actionable tasks 4. **Implementation Phase**: Execute tasks following established workflow patterns ## Artifact Relationships ### product.md - Defines WHAT and WHY Purpose: Captures product vision, goals, target users, and business context. Contents: - Product name and one-line description - Problem statement and solution approach - Target user personas - Core features and capabilities - Success metrics and KPIs - Product roadmap (high-level) Update when: - Product vision or goals change - New major features are planned - Target audience shifts - Business priorities evolve ### product-guidelines.md - Defines HOW to Communicate Purpose: Establishes brand voice, messaging standards, and communication patterns. Contents: - Brand voice and tone guidelines - Terminology and glossary - Error message conventions - User-facing copy standards - Documentation style Update when: - Brand guidelines change - New terminology is introduced - Communication patterns need refinement ### tech-stack.md - Defines WITH WHAT Purpose: Documents technology choices, dependencies, and architectural decisions. Contents: - Primary languages and frameworks - Key dependencies with versions - Infrastructure and deployment targets - Development tools and environment - Testing frameworks - Code quality tools Update when: - Adding new dependencies - Upgrading major versions - Changing infrastructure - Adopting new tools or patterns ### workflow.md - Defines HOW to Work Purpose: Establishes development practices, quality gates, and team workflows. Contents: - Development methodology (TDD, etc.) - Git workflow and commit conventions - Code review requirements - Testing requirements and coverage targets - Quality assurance gates - Deployment procedures Update when: - Team practices evolve - Quality standards change - New workflow patterns are adopted ### tracks.md - Tracks WHAT'S HAPPENING Purpose: Registry of all work units with status and metadata. Contents: - Active tracks with current status - Completed tracks with completion dates - Track metadata (type, priority, assignee) - Links to individual track directories Update when: - New tracks are created - Track status changes - Tracks are completed or archived See [references/artifact-templates.md](references/artifact-templates.md) for copy-paste starter templates. ## Context Maintenance Principles ### Keep Artifacts Synchronized Ensure changes in one artifact reflect in related documents: - New feature in product.md β†’ Update tech-stack.md if new dependencies needed - Completed track β†’ Update product.md to reflect new capabilities - Workflow change β†’ Update all affected track plans ### Update tech-stack.md When Adding Dependencies Before adding any new dependency: 1. Check if existing dependencies solve the need 2. Document the rationale for new dependencies 3. Add version constraints 4. Note any configuration requirements ### Update product.md When Features Complete After completing a feature track: 1. Move feature from "planned" to "implemented" in product.md 2. Update any affected success metrics 3. Document any scope changes from original plan ### Verify Context Before Implementation Before starting any track: 1. Read all context artifacts 2. Flag any outdated information 3. Propose updates before proceeding 4. Confirm context accuracy with stakeholders ## Greenfield vs Brownfield Handling ### Greenfield Projects (New) For new projects: 1. Run `/conductor:setup` to create all artifacts interactively 2. Answer questions about product vision, tech preferences, and workflow 3. Generate initial style guides for chosen languages 4. Create empty tracks registry Characteristics: - Full control over context structure - Define standards before code exists - Establish patterns early ### Brownfield Projects (Existing) For existing codebases: 1. Run `/conductor:setup` with existing codebase detection 2. System analyzes existing code, configs, and documentation 3. Pre-populate artifacts based on discovered patterns 4. Review and refine generated context Characteristics: - Extract implicit context from existing code - Reconcile existing patterns with desired patterns - Document technical debt and modernization plans - Preserve working patterns while establishing standards ## Benefits ### Team Alignment - New team members onboard faster with explicit context - Consistent terminology and conventions across the team - Shared understanding of product goals and technical decisions ### AI Consistency - AI assistants produce aligned outputs across sessions - Reduced need to re-explain context in each interaction - Predictable behavior based on documented standards ### Institutional Memory - Decisions and rationale are preserved - Context survives team changes - Historical context informs future decisions ### Quality Assurance - Standards are explicit and verifiable - Deviations from context are detectable - Quality gates are documented and enforceable ## Directory Structure ``` conductor/ β”œβ”€β”€ index.md # Navigation hub linking all artifacts β”œβ”€β”€ product.md # Product vision and goals β”œβ”€β”€ product-guidelines.md # Communication standards β”œβ”€β”€ tech-stack.md # Technology preferences β”œβ”€β”€ workflow.md # Development practices β”œβ”€β”€ tracks.md # Work unit registry β”œβ”€β”€ setup_state.json # Resumable setup state β”œβ”€β”€ code_styleguides/ # Language-specific conventions β”‚ β”œβ”€β”€ python.md β”‚ β”œβ”€β”€ typescript.md β”‚ └── ... └── tracks/ └── <track-id>/ β”œβ”€β”€ spec.md β”œβ”€β”€ plan.md β”œβ”€β”€ metadata.json └── index.md ``` ## Context Lifecycle 1. **Creation**: Initial setup via `/conductor:setup` 2. **Validation**: Verify before each track 3. **Evolution**: Update as project grows 4. **Synchronization**: Keep artifacts aligned 5. **Archival**: Document historical decisions ## Context Validation Checklist Before starting implementation on any track, validate context: ### Product Context - [ ] product.md reflects current product vision - [ ] Target users are accurately described - [ ] Feature list is up to date - [ ] Success metrics are defined ### Technical Context - [ ] tech-stack.md lists all current dependencies - [ ] Version numbers are accurate - [ ] Infrastructure targets are correct - [ ] Development tools are documented ### Workflow Context - [ ] workflow.md describes current practices - [ ] Quality gates are defined - [ ] Coverage targets are specified - [ ] Commit conventions are documented ### Track Context - [ ] tracks.md shows all active work - [ ] No stale or abandoned tracks - [ ] Dependencies between tracks are noted ## Common Anti-Patterns Avoid these context management mistakes: ### Stale Context Problem: Context documents become outdated and misleading. Solution: Update context as part of each track's completion process. ### Context Sprawl Problem: Information scattered across multiple locations. Solution: Use the defined artifact structure; resist creating new document types. ### Implicit Context Problem: Relying on knowledge not captured in artifacts. Solution: If you reference something repeatedly, add it to the appropriate artifact. ### Context Hoarding Problem: One person maintains context without team input. Solution: Review context artifacts in pull requests; make updates collaborative. ### Over-Specification Problem: Context becomes so detailed it's impossible to maintain. Solution: Keep artifacts focused on decisions that affect AI behavior and team alignment. ## Integration with Development Tools ### IDE Integration Configure your IDE to display context files prominently: - Pin conductor/product.md for quick reference - Add tech-stack.md to project notes - Create snippets for common patterns from style guides ### Git Hooks Consider pre-commit hooks that: - Warn when dependencies change without tech-stack.md update - Remind to update product.md when feature branches merge - Validate context artifact syntax ### CI/CD Integration Include context validation in pipelines: - Check tech-stack.md matches actual dependencies - Verify links in context documents resolve - Ensure tracks.md status matches git branch state ## Session Continuity Conductor supports multi-session development through context persistence: ### Starting a New Session 1. Read index.md to orient yourself 2. Check tracks.md for active work 3. Review relevant track's plan.md for current task 4. Verify context artifacts are current ### Ending a Session 1. Update plan.md with current progress 2. Note any blockers or decisions made 3. Commit in-progress work with clear status 4. Update tracks.md if status changed ### Handling Interruptions If interrupted mid-task: 1. Mark task as `[~]` with note about stopping point 2. Commit work-in-progress to feature branch 3. Document any uncommitted decisions in plan.md ## Best Practices 1. **Read context first**: Always read relevant artifacts before starting work 2. **Small updates**: Make incremental context changes, not massive rewrites 3. **Link decisions**: Reference context when making implementation choices 4. **Version context**: Commit context changes alongside code changes 5. **Review context**: Include context artifact reviews in code reviews 6. **Validate regularly**: Run context validation checklist before major work 7. **Communicate changes**: Notify team when context artifacts change significantly 8. **Preserve history**: Use git to track context evolution over time 9. **Question staleness**: If context feels wrong, investigate and update 10. **Keep it actionable**: Every context item should inform a decision or behavior
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

postgresql-table-design

Design a PostgreSQL-specific schema. Covers best-practices, data

data
⭐1
# PostgreSQL Table Design ## Core Rules - Define a **PRIMARY KEY** for reference tables (users, orders, etc.). Not always needed for time-series/event/log data. When used, prefer `BIGINT GENERATED ALWAYS AS IDENTITY`; use `UUID` only when global uniqueness/opacity is needed. - **Normalize first (to 3NF)** to eliminate data redundancy and update anomalies; denormalize **only** for measured, high-ROI reads where join performance is proven problematic. Premature denormalization creates maintenance burden. - Add **NOT NULL** everywhere it’s semantically required; use **DEFAULT**s for common values. - Create **indexes for access paths you actually query**: PK/unique (auto), **FK columns (manual!)**, frequent filters/sorts, and join keys. - Prefer **TIMESTAMPTZ** for event time; **NUMERIC** for money; **TEXT** for strings; **BIGINT** for integer values, **DOUBLE PRECISION** for floats (or `NUMERIC` for exact decimal arithmetic). ## PostgreSQL β€œGotchas” - **Identifiers**: unquoted β†’ lowercased. Avoid quoted/mixed-case names. Convention: use `snake_case` for table/column names. - **Unique + NULLs**: UNIQUE allows multiple NULLs. Use `UNIQUE (...) NULLS NOT DISTINCT` (PG15+) to restrict to one NULL. - **FK indexes**: PostgreSQL **does not** auto-index FK columns. Add them. - **No silent coercions**: length/precision overflows error out (no truncation). Example: inserting 999 into `NUMERIC(2,0)` fails with error, unlike some databases that silently truncate or round. - **Sequences/identity have gaps** (normal; don't "fix"). Rollbacks, crashes, and concurrent transactions create gaps in ID sequences (1, 2, 5, 6...). This is expected behaviorβ€”don't try to make IDs consecutive. - **Heap storage**: no clustered PK by default (unlike SQL Server/MySQL InnoDB); `CLUSTER` is one-off reorganization, not maintained on subsequent inserts. Row order on disk is insertion order unless explicitly clustered. - **MVCC**: updates/deletes leave dead tuples; vacuum handles themβ€”design to avoid hot wide-row churn. ## Data Types - **IDs**: `BIGINT GENERATED ALWAYS AS IDENTITY` preferred (`GENERATED BY DEFAULT` also fine); `UUID` when merging/federating/used in a distributed system or for opaque IDs. Generate with `uuidv7()` (preferred if using PG18+) or `gen_random_uuid()` (if using an older PG version). - **Integers**: prefer `BIGINT` unless storage space is critical; `INTEGER` for smaller ranges; avoid `SMALLINT` unless constrained. - **Floats**: prefer `DOUBLE PRECISION` over `REAL` unless storage space is critical. Use `NUMERIC` for exact decimal arithmetic. - **Strings**: prefer `TEXT`; if length limits needed, use `CHECK (LENGTH(col) <= n)` instead of `VARCHAR(n)`; avoid `CHAR(n)`. Use `BYTEA` for binary data. Large strings/binary (>2KB default threshold) automatically stored in TOAST with compression. TOAST storage: `PLAIN` (no TOAST), `EXTENDED` (compress + out-of-line), `EXTERNAL` (out-of-line, no compress), `MAIN` (compress, keep in-line if possible). Default `EXTENDED` usually optimal. Control with `ALTER TABLE tbl ALTER COLUMN col SET STORAGE strategy` and `ALTER TABLE tbl SET (toast_tuple_target = 4096)` for threshold. Case-insensitive: for locale/accent handling use non-deterministic collations; for plain ASCII use expression indexes on `LOWER(col)` (preferred unless column needs case-insensitive PK/FK/UNIQUE) or `CITEXT`. - **Money**: `NUMERIC(p,s)` (never float). - **Time**: `TIMESTAMPTZ` for timestamps; `DATE` for date-only; `INTERVAL` for durations. Avoid `TIMESTAMP` (without timezone). Use `now()` for transaction start time, `clock_timestamp()` for current wall-clock time. - **Booleans**: `BOOLEAN` with `NOT NULL` constraint unless tri-state values are required. - **Enums**: `CREATE TYPE ... AS ENUM` for small, stable sets (e.g. US states, days of week). For business-logic-driven and evolving values (e.g. order statuses) β†’ use TEXT (or INT) + CHECK or lookup table. - **Arrays**: `TEXT[]`, `INTEGER[]`, etc. Use for ordered lists where you query elements. Index with **GIN** for containment (`@>`, `<@`) and overlap (`&&`) queries. Access: `arr[1]` (1-indexed), `arr[1:3]` (slicing). Good for tags, categories; avoid for relationsβ€”use junction tables instead. Literal syntax: `'{val1,val2}'` or `ARRAY[val1,val2]`. - **Range types**: `daterange`, `numrange`, `tstzrange` for intervals. Support overlap (`&&`), containment (`@>`), operators. Index with **GiST**. Good for scheduling, versioning, numeric ranges. Pick a bounds scheme and use it consistently; prefer `[)` (inclusive/exclusive) by default. - **Network types**: `INET` for IP addresses, `CIDR` for network ranges, `MACADDR` for MAC addresses. Support network operators (`<<`, `>>`, `&&`). - **Geometric types**: `POINT`, `LINE`, `POLYGON`, `CIRCLE` for 2D spatial data. Index with **GiST**. Consider **PostGIS** for advanced spatial features. - **Text search**: `TSVECTOR` for full-text search documents, `TSQUERY` for search queries. Index `tsvector` with **GIN**. Always specify language: `to_tsvector('english', col)` and `to_tsquery('english', 'query')`. Never use single-argument versions. This applies to both index expressions and queries. - **Domain types**: `CREATE DOMAIN email AS TEXT CHECK (VALUE ~ '^[^@]+@[^@]+$')` for reusable custom types with validation. Enforces constraints across tables. - **Composite types**: `CREATE TYPE address AS (street TEXT, city TEXT, zip TEXT)` for structured data within columns. Access with `(col).field` syntax. - **JSONB**: preferred over JSON; index with **GIN**. Use only for optional/semi-structured attrs. ONLY use JSON if the original ordering of the contents MUST be preserved. - **Vector types**: `vector` type by `pgvector` for vector similarity search for embeddings. ### Do not use the following data types - DO NOT use `timestamp` (without time zone); DO use `timestamptz` instead. - DO NOT use `char(n)` or `varchar(n)`; DO use `text` instead. - DO NOT use `money` type; DO use `numeric` instead. - DO NOT use `timetz` type; DO use `timestamptz` instead. - DO NOT use `timestamptz(0)` or any other precision specification; DO use `timestamptz` instead - DO NOT use `serial` type; DO use `generated always as identity` instead. ## Table Types - **Regular**: default; fully durable, logged. - **TEMPORARY**: session-scoped, auto-dropped, not logged. Faster for scratch work. - **UNLOGGED**: persistent but not crash-safe. Faster writes; good for caches/staging. ## Row-Level Security Enable with `ALTER TABLE tbl ENABLE ROW LEVEL SECURITY`. Create policies: `CREATE POLICY user_access ON orders FOR SELECT TO app_users USING (user_id = current_user_id())`. Built-in user-based access control at the row level. ## Constraints - **PK**: implicit UNIQUE + NOT NULL; creates a B-tree index. - **FK**: specify `ON DELETE/UPDATE` action (`CASCADE`, `RESTRICT`, `SET NULL`, `SET DEFAULT`). Add explicit index on referencing columnβ€”speeds up joins and prevents locking issues on parent deletes/updates. Use `DEFERRABLE INITIALLY DEFERRED` for circular FK dependencies checked at transaction end. - **UNIQUE**: creates a B-tree index; allows multiple NULLs unless `NULLS NOT DISTINCT` (PG15+). Standard behavior: `(1, NULL)` and `(1, NULL)` are allowed. With `NULLS NOT DISTINCT`: only one `(1, NULL)` allowed. Prefer `NULLS NOT DISTINCT` unless you specifically need duplicate NULLs. - **CHECK**: row-local constraints; NULL values pass the check (three-valued logic). Example: `CHECK (price > 0)` allows NULL prices. Combine with `NOT NULL` to enforce: `price NUMERIC NOT NULL CHECK (price > 0)`. - **EXCLUDE**: prevents overlapping values using operators. `EXCLUDE USING gist (room_id WITH =, booking_period WITH &&)` prevents double-booking rooms. Requires appropriate index type (often GiST). ## Indexing - **B-tree**: default for equality/range queries (`=`, `<`, `>`, `BETWEEN`, `ORDER BY`) - **Composite**: order mattersβ€”index used if equality on leftmost prefix (`WHERE a = ? AND b > ?` uses index on `(a,b)`, but `WHERE b = ?` does not). Put most selective/frequently filtered columns first. - **Covering**: `CREATE INDEX ON tbl (id) INCLUDE (name, email)` - includes non-key columns for index-only scans without visiting table. - **Partial**: for hot subsets (`WHERE status = 'active'` β†’ `CREATE INDEX ON tbl (user_id) WHERE status = 'active'`). Any query with `status = 'active'` can use this index. - **Expression**: for computed search keys (`CREATE INDEX ON tbl (LOWER(email))`). Expression must match exactly in WHERE clause: `WHERE LOWER(email) = 'user@example.com'`. - **GIN**: JSONB containment/existence, arrays (`@>`, `?`), full-text search (`@@`) - **GiST**: ranges, geometry, exclusion constraints - **BRIN**: very large, naturally ordered data (time-series)β€”minimal storage overhead. Effective when row order on disk correlates with indexed column (insertion order or after `CLUSTER`). ## Partitioning - Use for very large tables (>100M rows) where queries consistently filter on partition key (often time/date). - Alternate use: use for tables where data maintenance tasks dictates e.g. data pruned or bulk replaced periodically - **RANGE**: common for time-series (`PARTITION BY RANGE (created_at)`). Create partitions: `CREATE TABLE logs_2024_01 PARTITION OF logs FOR VALUES FROM ('2024-01-01') TO ('2024-02-01')`. **TimescaleDB** automates time-based or ID-based partitioning with retention policies and compression. - **LIST**: for discrete values (`PARTITION BY LIST (region)`). Example: `FOR VALUES IN ('us-east', 'us-west')`. - **HASH**: for even distribution when no natural key (`PARTITION BY HASH (user_id)`). Creates N partitions with modulus. - **Constraint exclusion**: requires `CHECK` constraints on partitions for query planner to prune. Auto-created for declarative partitioning (PG10+). - Prefer declarative partitioning or hypertables. Do NOT use table inheritance. - **Limitations**: no global UNIQUE constraintsβ€”include partition key in PK/UNIQUE. FKs from partitioned tables not supported; use triggers. ## Special Considerations ### Update-Heavy Tables - **Separate hot/cold columns**β€”put frequently updated columns in separate table to minimize bloat. - **Use `fillfactor=90`** to leave space for HOT updates that avoid index maintenance. - **Avoid updating indexed columns**β€”prevents beneficial HOT updates. - **Partition by update patterns**β€”separate frequently updated rows in a different partition from stable data. ### Insert-Heavy Workloads - **Minimize indexes**β€”only create what you query; every index slows inserts. - **Use `COPY` or multi-row `INSERT`** instead of single-row inserts. - **UNLOGGED tables** for rebuildable staging dataβ€”much faster writes. - **Defer index creation** for bulk loadsβ€”>drop index, load data, recreate indexes. - **Partition by time/hash** to distribute load. **TimescaleDB** automates partitioning and compression of insert-heavy data. - **Use a natural key for primary key** such as a (timestamp, device_id) if enforcing global uniqueness is important many insert-heavy tables don't need a primary key at all. - If you do need a surrogate key, **Prefer `BIGINT GENERATED ALWAYS AS IDENTITY` over `UUID`**. ### Upsert-Friendly Design - **Requires UNIQUE index** on conflict target columnsβ€”`ON CONFLICT (col1, col2)` needs exact matching unique index (partial indexes don't work). - **Use `EXCLUDED.column`** to reference would-be-inserted values; only update columns that actually changed to reduce write overhead. - **`DO NOTHING` faster** than `DO UPDATE` when no actual update needed. ### Safe Schema Evolution - **Transactional DDL**: most DDL operations can run in transactions and be rolled backβ€”`BEGIN; ALTER TABLE...; ROLLBACK;` for safe testing. - **Concurrent index creation**: `CREATE INDEX CONCURRENTLY` avoids blocking writes but can't run in transactions. - **Volatile defaults cause rewrites**: adding `NOT NULL` columns with volatile defaults (e.g., `now()`, `gen_random_uuid()`) rewrites entire table. Non-volatile defaults are fast. - **Drop constraints before columns**: `ALTER TABLE DROP CONSTRAINT` then `DROP COLUMN` to avoid dependency issues. - **Function signature changes**: `CREATE OR REPLACE` with different arguments creates overloads, not replacements. DROP old version if no overload desired. ## Generated Columns - `... GENERATED ALWAYS AS (<expr>) STORED` for computed, indexable fields. PG18+ adds `VIRTUAL` columns (computed on read, not stored). ## Extensions - **`pgcrypto`**: `crypt()` for password hashing. - **`uuid-ossp`**: alternative UUID functions; prefer `pgcrypto` for new projects. - **`pg_trgm`**: fuzzy text search with `%` operator, `similarity()` function. Index with GIN for `LIKE '%pattern%'` acceleration. - **`citext`**: case-insensitive text type. Prefer expression indexes on `LOWER(col)` unless you need case-insensitive constraints. - **`btree_gin`/`btree_gist`**: enable mixed-type indexes (e.g., GIN index on both JSONB and text columns). - **`hstore`**: key-value pairs; mostly superseded by JSONB but useful for simple string mappings. - **`timescaledb`**: essential for time-seriesβ€”automated partitioning, retention, compression, continuous aggregates. - **`postgis`**: comprehensive geospatial support beyond basic geometric typesβ€”essential for location-based applications. - **`pgvector`**: vector similarity search for embeddings. - **`pgaudit`**: audit logging for all database activity. ## JSONB Guidance - Prefer `JSONB` with **GIN** index. - Default: `CREATE INDEX ON tbl USING GIN (jsonb_col);` β†’ accelerates: - **Containment** `jsonb_col @> '{"k":"v"}'` - **Key existence** `jsonb_col ? 'k'`, **any/all keys** `?\|`, `?&` - **Path containment** on nested docs - **Disjunction** `jsonb_col @> ANY(ARRAY['{"status":"active"}', '{"status":"pending"}'])` - Heavy `@>` workloads: consider opclass `jsonb_path_ops` for smaller/faster containment-only indexes: - `CREATE INDEX ON tbl USING GIN (jsonb_col jsonb_path_ops);` - **Trade-off**: loses support for key existence (`?`, `?|`, `?&`) queriesβ€”only supports containment (`@>`) - Equality/range on a specific scalar field: extract and index with B-tree (generated column or expression): - `ALTER TABLE tbl ADD COLUMN price INT GENERATED ALWAYS AS ((jsonb_col->>'price')::INT) STORED;` - `CREATE INDEX ON tbl (price);` - Prefer queries like `WHERE price BETWEEN 100 AND 500` (uses B-tree) over `WHERE (jsonb_col->>'price')::INT BETWEEN 100 AND 500` without index. - Arrays inside JSONB: use GIN + `@>` for containment (e.g., tags). Consider `jsonb_path_ops` if only doing containment. - Keep core relations in tables; use JSONB for optional/variable attributes. - Use constraints to limit allowed JSONB values in a column e.g. `config JSONB NOT NULL CHECK(jsonb_typeof(config) = 'object')` ## Examples ### Users ```sql CREATE TABLE users ( user_id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY, email TEXT NOT NULL UNIQUE, name TEXT NOT NULL, created_at TIMESTAMPTZ NOT NULL DEFAULT now() ); CREATE UNIQUE INDEX ON users (LOWER(email)); CREATE INDEX ON users (created_at); ``` ### Orders ```sql CREATE TABLE orders ( order_id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY, user_id BIGINT NOT NULL REFERENCES users(user_id), status TEXT NOT NULL DEFAULT 'PENDING' CHECK (status IN ('PENDING','PAID','CANCELED')), total NUMERIC(10,2) NOT NULL CHECK (total > 0), created_at TIMESTAMPTZ NOT NULL DEFAULT now() ); CREATE INDEX ON orders (user_id); CREATE INDEX ON orders (created_at); ``` ### JSONB ```sql CREATE TABLE profiles ( user_id BIGINT PRIMARY KEY REFERENCES users(user_id), attrs JSONB NOT NULL DEFAULT '{}', theme TEXT GENERATED ALWAYS AS (attrs->>'theme') STORED ); CREATE INDEX profiles_attrs_gin ON profiles USING GIN (attrs); ```
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

parallel-feature-development

Coordinate parallel feature development with file ownership

coding
⭐1
# Parallel Feature Development Strategies for decomposing features into parallel work streams, establishing file ownership boundaries, avoiding conflicts, and integrating results from multiple implementer agents. ## When to Use This Skill - Decomposing a feature for parallel implementation - Establishing file ownership boundaries between agents - Designing interface contracts between parallel work streams - Choosing integration strategies (vertical slice vs horizontal layer) - Managing branch and merge workflows for parallel development ## File Ownership Strategies ### By Directory Assign each implementer ownership of specific directories: ``` implementer-1: src/components/auth/ implementer-2: src/api/auth/ implementer-3: tests/auth/ ``` **Best for**: Well-organized codebases with clear directory boundaries. ### By Module Assign ownership of logical modules (which may span directories): ``` implementer-1: Authentication module (login, register, logout) implementer-2: Authorization module (roles, permissions, guards) ``` **Best for**: Feature-oriented architectures, domain-driven design. ### By Layer Assign ownership of architectural layers: ``` implementer-1: UI layer (components, styles, layouts) implementer-2: Business logic layer (services, validators) implementer-3: Data layer (models, repositories, migrations) ``` **Best for**: Traditional MVC/layered architectures. ## Conflict Avoidance Rules ### The Cardinal Rule **One owner per file.** No file should be assigned to multiple implementers. ### When Files Must Be Shared If a file genuinely needs changes from multiple implementers: 1. **Designate a single owner** β€” One implementer owns the file 2. **Other implementers request changes** β€” Message the owner with specific change requests 3. **Owner applies changes sequentially** β€” Prevents merge conflicts 4. **Alternative: Extract interfaces** β€” Create a separate interface file that the non-owner can import without modifying ### Interface Contracts When implementers need to coordinate at boundaries: ```typescript // src/types/auth-contract.ts (owned by team-lead, read-only for implementers) export interface AuthResponse { token: string; user: UserProfile; expiresAt: number; } export interface AuthService { login(email: string, password: string): Promise<AuthResponse>; register(data: RegisterData): Promise<AuthResponse>; } ``` Both implementers import from the contract file but neither modifies it. ## Integration Patterns ### Vertical Slice Each implementer builds a complete feature slice (UI + API + tests): ``` implementer-1: Login feature (login form + login API + login tests) implementer-2: Register feature (register form + register API + register tests) ``` **Pros**: Each slice is independently testable, minimal integration needed. **Cons**: May duplicate shared utilities, harder with tightly coupled features. ### Horizontal Layer Each implementer builds one layer across all features: ``` implementer-1: All UI components (login form, register form, profile page) implementer-2: All API endpoints (login, register, profile) implementer-3: All tests (unit, integration, e2e) ``` **Pros**: Consistent patterns within each layer, natural specialization. **Cons**: More integration points, layer 3 depends on layers 1 and 2. ### Hybrid Mix vertical and horizontal based on coupling: ``` implementer-1: Login feature (vertical slice β€” UI + API + tests) implementer-2: Shared auth infrastructure (horizontal β€” middleware, JWT utils, types) ``` **Best for**: Most real-world features with some shared infrastructure. ## Branch Management ### Single Branch Strategy All implementers work on the same feature branch: - Simple setup, no merge overhead - Requires strict file ownership to avoid conflicts - Best for: small teams (2-3), well-defined boundaries ### Multi-Branch Strategy Each implementer works on a sub-branch: ``` feature/auth β”œβ”€β”€ feature/auth-login (implementer-1) β”œβ”€β”€ feature/auth-register (implementer-2) └── feature/auth-tests (implementer-3) ``` - More isolation, explicit merge points - Higher overhead, merge conflicts still possible in shared files - Best for: larger teams (4+), complex features
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

javascript-testing-patterns

Implement comprehensive testing strategies using Jest, Vitest, and

coding
⭐1
# JavaScript Testing Patterns Comprehensive guide for implementing robust testing strategies in JavaScript/TypeScript applications using modern testing frameworks and best practices. ## When to Use This Skill - Setting up test infrastructure for new projects - Writing unit tests for functions and classes - Creating integration tests for APIs and services - Implementing end-to-end tests for user flows - Mocking external dependencies and APIs - Testing React, Vue, or other frontend components - Implementing test-driven development (TDD) - Setting up continuous testing in CI/CD pipelines ## Testing Frameworks ### Jest - Full-Featured Testing Framework **Setup:** ```typescript // jest.config.ts import type { Config } from "jest"; const config: Config = { preset: "ts-jest", testEnvironment: "node", roots: ["<rootDir>/src"], testMatch: ["**/__tests__/**/*.ts", "**/?(*.)+(spec|test).ts"], collectCoverageFrom: [ "src/**/*.ts", "!src/**/*.d.ts", "!src/**/*.interface.ts", ], coverageThreshold: { global: { branches: 80, functions: 80, lines: 80, statements: 80, }, }, setupFilesAfterEnv: ["<rootDir>/src/test/setup.ts"], }; export default config; ``` ### Vitest - Fast, Vite-Native Testing **Setup:** ```typescript // vitest.config.ts import { defineConfig } from "vitest/config"; export default defineConfig({ test: { globals: true, environment: "node", coverage: { provider: "v8", reporter: ["text", "json", "html"], exclude: ["**/*.d.ts", "**/*.config.ts", "**/dist/**"], }, setupFiles: ["./src/test/setup.ts"], }, }); ``` ## Unit Testing Patterns ### Pattern 1: Testing Pure Functions ```typescript // utils/calculator.ts export function add(a: number, b: number): number { return a + b; } export function divide(a: number, b: number): number { if (b === 0) { throw new Error("Division by zero"); } return a / b; } // utils/calculator.test.ts import { describe, it, expect } from "vitest"; import { add, divide } from "./calculator"; describe("Calculator", () => { describe("add", () => { it("should add two positive numbers", () => { expect(add(2, 3)).toBe(5); }); it("should add negative numbers", () => { expect(add(-2, -3)).toBe(-5); }); it("should handle zero", () => { expect(add(0, 5)).toBe(5); expect(add(5, 0)).toBe(5); }); }); describe("divide", () => { it("should divide two numbers", () => { expect(divide(10, 2)).toBe(5); }); it("should handle decimal results", () => { expect(divide(5, 2)).toBe(2.5); }); it("should throw error when dividing by zero", () => { expect(() => divide(10, 0)).toThrow("Division by zero"); }); }); }); ``` ### Pattern 2: Testing Classes ```typescript // services/user.service.ts export class UserService { private users: Map<string, User> = new Map(); create(user: User): User { if (this.users.has(user.id)) { throw new Error("User already exists"); } this.users.set(user.id, user); return user; } findById(id: string): User | undefined { return this.users.get(id); } update(id: string, updates: Partial<User>): User { const user = this.users.get(id); if (!user) { throw new Error("User not found"); } const updated = { ...user, ...updates }; this.users.set(id, updated); return updated; } delete(id: string): boolean { return this.users.delete(id); } } // services/user.service.test.ts import { describe, it, expect, beforeEach } from "vitest"; import { UserService } from "./user.service"; describe("UserService", () => { let service: UserService; beforeEach(() => { service = new UserService(); }); describe("create", () => { it("should create a new user", () => { const user = { id: "1", name: "John", email: "john@example.com" }; const created = service.create(user); expect(created).toEqual(user); expect(service.findById("1")).toEqual(user); }); it("should throw error if user already exists", () => { const user = { id: "1", name: "John", email: "john@example.com" }; service.create(user); expect(() => service.create(user)).toThrow("User already exists"); }); }); describe("update", () => { it("should update existing user", () => { const user = { id: "1", name: "John", email: "john@example.com" }; service.create(user); const updated = service.update("1", { name: "Jane" }); expect(updated.name).toBe("Jane"); expect(updated.email).toBe("john@example.com"); }); it("should throw error if user not found", () => { expect(() => service.update("999", { name: "Jane" })).toThrow( "User not found", ); }); }); }); ``` ### Pattern 3: Testing Async Functions ```typescript // services/api.service.ts export class ApiService { async fetchUser(id: string): Promise<User> { const response = await fetch(`https://api.example.com/users/${id}`); if (!response.ok) { throw new Error("User not found"); } return response.json(); } async createUser(user: CreateUserDTO): Promise<User> { const response = await fetch("https://api.example.com/users", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify(user), }); return response.json(); } } // services/api.service.test.ts import { describe, it, expect, vi, beforeEach } from "vitest"; import { ApiService } from "./api.service"; // Mock fetch globally global.fetch = vi.fn(); describe("ApiService", () => { let service: ApiService; beforeEach(() => { service = new ApiService(); vi.clearAllMocks(); }); describe("fetchUser", () => { it("should fetch user successfully", async () => { const mockUser = { id: "1", name: "John", email: "john@example.com" }; (fetch as any).mockResolvedValueOnce({ ok: true, json: async () => mockUser, }); const user = await service.fetchUser("1"); expect(user).toEqual(mockUser); expect(fetch).toHaveBeenCalledWith("https://api.example.com/users/1"); }); it("should throw error if user not found", async () => { (fetch as any).mockResolvedValueOnce({ ok: false, }); await expect(service.fetchUser("999")).rejects.toThrow("User not found"); }); }); describe("createUser", () => { it("should create user successfully", async () => { const newUser = { name: "John", email: "john@example.com" }; const createdUser = { id: "1", ...newUser }; (fetch as any).mockResolvedValueOnce({ ok: true, json: async () => createdUser, }); const user = await service.createUser(newUser); expect(user).toEqual(createdUser); expect(fetch).toHaveBeenCalledWith( "https://api.example.com/users", expect.objectContaining({ method: "POST", body: JSON.stringify(newUser), }), ); }); }); }); ``` ## Mocking Patterns ### Pattern 1: Mocking Modules ```typescript // services/email.service.ts import nodemailer from "nodemailer"; export class EmailService { private transporter = nodemailer.createTransport({ host: process.env.SMTP_HOST, port: 587, auth: { user: process.env.SMTP_USER, pass: process.env.SMTP_PASS, }, }); async sendEmail(to: string, subject: string, html: string) { await this.transporter.sendMail({ from: process.env.EMAIL_FROM, to, subject, html, }); } } // services/email.service.test.ts import { describe, it, expect, vi, beforeEach } from "vitest"; import { EmailService } from "./email.service"; vi.mock("nodemailer", () => ({ default: { createTransport: vi.fn(() => ({ sendMail: vi.fn().mockResolvedValue({ messageId: "123" }), })), }, })); describe("EmailService", () => { let service: EmailService; beforeEach(() => { service = new EmailService(); }); it("should send email successfully", async () => { await service.sendEmail( "test@example.com", "Test Subject", "<p>Test Body</p>", ); expect(service["transporter"].sendMail).toHaveBeenCalledWith( expect.objectContaining({ to: "test@example.com", subject: "Test Subject", }), ); }); }); ``` ### Pattern 2: Dependency Injection for Testing ```typescript // services/user.service.ts export interface IUserRepository { findById(id: string): Promise<User | null>; create(user: User): Promise<User>; } export class UserService { constructor(private userRepository: IUserRepository) {} async getUser(id: string): Promise<User> { const user = await this.userRepository.findById(id); if (!user) { throw new Error("User not found"); } return user; } async createUser(userData: CreateUserDTO): Promise<User> { // Business logic here const user = { id: generateId(), ...userData }; return this.userRepository.create(user); } } // services/user.service.test.ts import { describe, it, expect, vi, beforeEach } from "vitest"; import { UserService, IUserRepository } from "./user.service"; describe("UserService", () => { let service: UserService; let mockRepository: IUserRepository; beforeEach(() => { mockRepository = { findById: vi.fn(), create: vi.fn(), }; service = new UserService(mockRepository); }); describe("getUser", () => { it("should return user if found", async () => { const mockUser = { id: "1", name: "John", email: "john@example.com" }; vi.mocked(mockRepository.findById).mockResolvedValue(mockUser); const user = await service.getUser("1"); expect(user).toEqual(mockUser); expect(mockRepository.findById).toHaveBeenCalledWith("1"); }); it("should throw error if user not found", async () => { vi.mocked(mockRepository.findById).mockResolvedValue(null); await expect(service.getUser("999")).rejects.toThrow("User not found"); }); }); describe("createUser", () => { it("should create user successfully", async () => { const userData = { name: "John", email: "john@example.com" }; const createdUser = { id: "1", ...userData }; vi.mocked(mockRepository.create).mockResolvedValue(createdUser); const user = await service.createUser(userData); expect(user).toEqual(createdUser); expect(mockRepository.create).toHaveBeenCalled(); }); }); }); ``` ### Pattern 3: Spying on Functions ```typescript // utils/logger.ts export const logger = { info: (message: string) => console.log(`INFO: ${message}`), error: (message: string) => console.error(`ERROR: ${message}`), }; // services/order.service.ts import { logger } from "../utils/logger"; export class OrderService { async processOrder(orderId: string): Promise<void> { logger.info(`Processing order ${orderId}`); // Process order logic logger.info(`Order ${orderId} processed successfully`); } } // services/order.service.test.ts import { describe, it, expect, vi, beforeEach, afterEach } from "vitest"; import { OrderService } from "./order.service"; import { logger } from "../utils/logger"; describe("OrderService", () => { let service: OrderService; let loggerSpy: any; beforeEach(() => { service = new OrderService(); loggerSpy = vi.spyOn(logger, "info"); }); afterEach(() => { loggerSpy.mockRestore(); }); it("should log order processing", async () => { await service.processOrder("123"); expect(loggerSpy).toHaveBeenCalledWith("Processing order 123"); expect(loggerSpy).toHaveBeenCalledWith("Order 123 processed successfully"); expect(loggerSpy).toHaveBeenCalledTimes(2); }); }); ``` ## Integration Testing ### Pattern 1: API Integration Tests ```typescript // tests/integration/user.api.test.ts import request from "supertest"; import { app } from "../../src/app"; import { pool } from "../../src/config/database"; describe("User API Integration Tests", () => { beforeAll(async () => { // Setup test database await pool.query("CREATE TABLE IF NOT EXISTS users (...)"); }); afterAll(async () => { // Cleanup await pool.query("DROP TABLE IF EXISTS users"); await pool.end(); }); beforeEach(async () => { // Clear data before each test await pool.query("TRUNCATE TABLE users CASCADE"); }); describe("POST /api/users", () => { it("should create a new user", async () => { const userData = { name: "John Doe", email: "john@example.com", password: "password123", }; const response = await request(app) .post("/api/users") .send(userData) .expect(201); expect(response.body).toMatchObject({ name: userData.name, email: userData.email, }); expect(response.body).toHaveProperty("id"); expect(response.body).not.toHaveProperty("password"); }); it("should return 400 if email is invalid", async () => { const userData = { name: "John Doe", email: "invalid-email", password: "password123", }; const response = await request(app) .post("/api/users") .send(userData) .expect(400); expect(response.body).toHaveProperty("error"); }); it("should return 409 if email already exists", async () => { const userData = { name: "John Doe", email: "john@example.com", password: "password123", }; await request(app).post("/api/users").send(userData); const response = await request(app) .post("/api/users") .send(userData) .expect(409); expect(response.body.error).toContain("already exists"); }); }); describe("GET /api/users/:id", () => { it("should get user by id", async () => { const createResponse = await request(app).post("/api/users").send({ name: "John Doe", email: "john@example.com", password: "password123", }); const userId = createResponse.body.id; const response = await request(app) .get(`/api/users/${userId}`) .expect(200); expect(response.body).toMatchObject({ id: userId, name: "John Doe", email: "john@example.com", }); }); it("should return 404 if user not found", async () => { await request(app).get("/api/users/999").expect(404); }); }); describe("Authentication", () => { it("should require authentication for protected routes", async () => { await request(app).get("/api/users/me").expect(401); }); it("should allow access with valid token", async () => { // Create user and login await request(app).post("/api/users").send({ name: "John Doe", email: "john@example.com", password: "password123", }); const loginResponse = await request(app).post("/api/auth/login").send({ email: "john@example.com", password: "password123", }); const token = loginResponse.body.token; const response = await request(app) .get("/api/users/me") .set("Authorization", `Bearer ${token}`) .expect(200); expect(response.body.email).toBe("john@example.com"); }); }); }); ``` ### Pattern 2: Database Integration Tests ```typescript // tests/integration/user.repository.test.ts import { describe, it, expect, beforeAll, afterAll, beforeEach } from "vitest"; import { Pool } from "pg"; import { UserRepository } from "../../src/repositories/user.repository"; describe("UserRepository Integration Tests", () => { let pool: Pool; let repository: UserRepository; beforeAll(async () => { pool = new Pool({ host: "localhost", port: 5432, database: "test_db", user: "test_user", password: "test_password", }); repository = new UserRepository(pool); // Create tables await pool.query(` CREATE TABLE IF NOT EXISTS users ( id SERIAL PRIMARY KEY, name VARCHAR(255) NOT NULL, email VARCHAR(255) UNIQUE NOT NULL, password VARCHAR(255) NOT NULL, created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ) `); }); afterAll(async () => { await pool.query("DROP TABLE IF EXISTS users"); await pool.end(); }); beforeEach(async () => { await pool.query("TRUNCATE TABLE users CASCADE"); }); it("should create a user", async () => { const user = await repository.create({ name: "John Doe", email: "john@example.com", password: "hashed_password", }); expect(user).toHaveProperty("id"); expect(user.name).toBe("John Doe"); expect(user.email).toBe("john@example.com"); }); it("should find user by email", async () => { await repository.create({ name: "John Doe", email: "john@example.com", password: "hashed_password", }); const user = await repository.findByEmail("john@example.com"); expect(user).toBeTruthy(); expect(user?.name).toBe("John Doe"); }); it("should return null if user not found", async () => { const user = await repository.findByEmail("nonexistent@example.com"); expect(user).toBeNull(); }); }); ``` ## Frontend Testing with Testing Library ### Pattern 1: React Component Testing ```typescript // components/UserForm.tsx import { useState } from 'react'; interface Props { onSubmit: (user: { name: string; email: string }) => void; } export function UserForm({ onSubmit }: Props) { const [name, setName] = useState(''); const [email, setEmail] = useState(''); const handleSubmit = (e: React.FormEvent) => { e.preventDefault(); onSubmit({ name, email }); }; return ( <form onSubmit={handleSubmit}> <input type="text" placeholder="Name" value={name} onChange={(e) => setName(e.target.value)} data-testid="name-input" /> <input type="email" placeholder="Email" value={email} onChange={(e) => setEmail(e.target.value)} data-testid="email-input" /> <button type="submit">Submit</button> </form> ); } // components/UserForm.test.tsx import { render, screen, fireEvent } from '@testing-library/react'; import { describe, it, expect, vi } from 'vitest'; import { UserForm } from './UserForm'; describe('UserForm', () => { it('should render form inputs', () => { render(<UserForm onSubmit={vi.fn()} />); expect(screen.getByPlaceholderText('Name')).toBeInTheDocument(); expect(screen.getByPlaceholderText('Email')).toBeInTheDocument(); expect(screen.getByRole('button', { name: 'Submit' })).toBeInTheDocument(); }); it('should update input values', () => { render(<UserForm onSubmit={vi.fn()} />); const nameInput = screen.getByTestId('name-input') as HTMLInputElement; const emailInput = screen.getByTestId('email-input') as HTMLInputElement; fireEvent.change(nameInput, { target: { value: 'John Doe' } }); fireEvent.change(emailInpu
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

python-background-jobs

Python background job patterns including task queues, workers, and

coding
⭐1
# Python Background Jobs & Task Queues Decouple long-running or unreliable work from request/response cycles. Return immediately to the user while background workers handle the heavy lifting asynchronously. ## When to Use This Skill - Processing tasks that take longer than a few seconds - Sending emails, notifications, or webhooks - Generating reports or exporting data - Processing uploads or media transformations - Integrating with unreliable external services - Building event-driven architectures ## Core Concepts ### 1. Task Queue Pattern API accepts request, enqueues a job, returns immediately with a job ID. Workers process jobs asynchronously. ### 2. Idempotency Tasks may be retried on failure. Design for safe re-execution. ### 3. Job State Machine Jobs transition through states: pending β†’ running β†’ succeeded/failed. ### 4. At-Least-Once Delivery Most queues guarantee at-least-once delivery. Your code must handle duplicates. ## Quick Start This skill uses Celery for examples, a widely adopted task queue. Alternatives like RQ, Dramatiq, and cloud-native solutions (AWS SQS, GCP Tasks) are equally valid choices. ```python from celery import Celery app = Celery("tasks", broker="redis://localhost:6379") @app.task def send_email(to: str, subject: str, body: str) -> None: # This runs in a background worker email_client.send(to, subject, body) # In your API handler send_email.delay("user@example.com", "Welcome!", "Thanks for signing up") ``` ## Fundamental Patterns ### Pattern 1: Return Job ID Immediately For operations exceeding a few seconds, return a job ID and process asynchronously. ```python from uuid import uuid4 from dataclasses import dataclass from enum import Enum from datetime import datetime class JobStatus(Enum): PENDING = "pending" RUNNING = "running" SUCCEEDED = "succeeded" FAILED = "failed" @dataclass class Job: id: str status: JobStatus created_at: datetime started_at: datetime | None = None completed_at: datetime | None = None result: dict | None = None error: str | None = None # API endpoint async def start_export(request: ExportRequest) -> JobResponse: """Start export job and return job ID.""" job_id = str(uuid4()) # Persist job record await jobs_repo.create(Job( id=job_id, status=JobStatus.PENDING, created_at=datetime.utcnow(), )) # Enqueue task for background processing await task_queue.enqueue( "export_data", job_id=job_id, params=request.model_dump(), ) # Return immediately with job ID return JobResponse( job_id=job_id, status="pending", poll_url=f"/jobs/{job_id}", ) ``` ### Pattern 2: Celery Task Configuration Configure Celery tasks with proper retry and timeout settings. ```python from celery import Celery app = Celery("tasks", broker="redis://localhost:6379") # Global configuration app.conf.update( task_time_limit=3600, # Hard limit: 1 hour task_soft_time_limit=3000, # Soft limit: 50 minutes task_acks_late=True, # Acknowledge after completion task_reject_on_worker_lost=True, worker_prefetch_multiplier=1, # Don't prefetch too many tasks ) @app.task( bind=True, max_retries=3, default_retry_delay=60, autoretry_for=(ConnectionError, TimeoutError), ) def process_payment(self, payment_id: str) -> dict: """Process payment with automatic retry on transient errors.""" try: result = payment_gateway.charge(payment_id) return {"status": "success", "transaction_id": result.id} except PaymentDeclinedError as e: # Don't retry permanent failures return {"status": "declined", "reason": str(e)} except TransientError as e: # Retry with exponential backoff raise self.retry(exc=e, countdown=2 ** self.request.retries * 60) ``` ### Pattern 3: Make Tasks Idempotent Workers may retry on crash or timeout. Design for safe re-execution. ```python @app.task(bind=True) def process_order(self, order_id: str) -> None: """Process order idempotently.""" order = orders_repo.get(order_id) # Already processed? Return early if order.status == OrderStatus.COMPLETED: logger.info("Order already processed", order_id=order_id) return # Already in progress? Check if we should continue if order.status == OrderStatus.PROCESSING: # Use idempotency key to avoid double-charging pass # Process with idempotency key result = payment_provider.charge( amount=order.total, idempotency_key=f"order-{order_id}", # Critical! ) orders_repo.update(order_id, status=OrderStatus.COMPLETED) ``` **Idempotency Strategies:** 1. **Check-before-write**: Verify state before action 2. **Idempotency keys**: Use unique tokens with external services 3. **Upsert patterns**: `INSERT ... ON CONFLICT UPDATE` 4. **Deduplication window**: Track processed IDs for N hours ### Pattern 4: Job State Management Persist job state transitions for visibility and debugging. ```python class JobRepository: """Repository for managing job state.""" async def create(self, job: Job) -> Job: """Create new job record.""" await self._db.execute( """INSERT INTO jobs (id, status, created_at) VALUES ($1, $2, $3)""", job.id, job.status.value, job.created_at, ) return job async def update_status( self, job_id: str, status: JobStatus, **fields, ) -> None: """Update job status with timestamp.""" updates = {"status": status.value, **fields} if status == JobStatus.RUNNING: updates["started_at"] = datetime.utcnow() elif status in (JobStatus.SUCCEEDED, JobStatus.FAILED): updates["completed_at"] = datetime.utcnow() await self._db.execute( "UPDATE jobs SET status = $1, ... WHERE id = $2", updates, job_id, ) logger.info( "Job status updated", job_id=job_id, status=status.value, ) ``` ## Advanced Patterns ### Pattern 5: Dead Letter Queue Handle permanently failed tasks for manual inspection. ```python @app.task(bind=True, max_retries=3) def process_webhook(self, webhook_id: str, payload: dict) -> None: """Process webhook with DLQ for failures.""" try: result = send_webhook(payload) if not result.success: raise WebhookFailedError(result.error) except Exception as e: if self.request.retries >= self.max_retries: # Move to dead letter queue for manual inspection dead_letter_queue.send({ "task": "process_webhook", "webhook_id": webhook_id, "payload": payload, "error": str(e), "attempts": self.request.retries + 1, "failed_at": datetime.utcnow().isoformat(), }) logger.error( "Webhook moved to DLQ after max retries", webhook_id=webhook_id, error=str(e), ) return # Exponential backoff retry raise self.retry(exc=e, countdown=2 ** self.request.retries * 60) ``` ### Pattern 6: Status Polling Endpoint Provide an endpoint for clients to check job status. ```python from fastapi import FastAPI, HTTPException app = FastAPI() @app.get("/jobs/{job_id}") async def get_job_status(job_id: str) -> JobStatusResponse: """Get current status of a background job.""" job = await jobs_repo.get(job_id) if job is None: raise HTTPException(404, f"Job {job_id} not found") return JobStatusResponse( job_id=job.id, status=job.status.value, created_at=job.created_at, started_at=job.started_at, completed_at=job.completed_at, result=job.result if job.status == JobStatus.SUCCEEDED else None, error=job.error if job.status == JobStatus.FAILED else None, # Helpful for clients is_terminal=job.status in (JobStatus.SUCCEEDED, JobStatus.FAILED), ) ``` ### Pattern 7: Task Chaining and Workflows Compose complex workflows from simple tasks. ```python from celery import chain, group, chord # Simple chain: A β†’ B β†’ C workflow = chain( extract_data.s(source_id), transform_data.s(), load_data.s(destination_id), ) # Parallel execution: A, B, C all at once parallel = group( send_email.s(user_email), send_sms.s(user_phone), update_analytics.s(event_data), ) # Chord: Run tasks in parallel, then a callback # Process all items, then send completion notification workflow = chord( [process_item.s(item_id) for item_id in item_ids], send_completion_notification.s(batch_id), ) workflow.apply_async() ``` ### Pattern 8: Alternative Task Queues Choose the right tool for your needs. **RQ (Redis Queue)**: Simple, Redis-based ```python from rq import Queue from redis import Redis queue = Queue(connection=Redis()) job = queue.enqueue(send_email, "user@example.com", "Subject", "Body") ``` **Dramatiq**: Modern Celery alternative ```python import dramatiq from dramatiq.brokers.redis import RedisBroker dramatiq.set_broker(RedisBroker()) @dramatiq.actor def send_email(to: str, subject: str, body: str) -> None: email_client.send(to, subject, body) ``` **Cloud-native options:** - AWS SQS + Lambda - Google Cloud Tasks - Azure Functions ## Best Practices Summary 1. **Return immediately** - Don't block requests for long operations 2. **Persist job state** - Enable status polling and debugging 3. **Make tasks idempotent** - Safe to retry on any failure 4. **Use idempotency keys** - For external service calls 5. **Set timeouts** - Both soft and hard limits 6. **Implement DLQ** - Capture permanently failed tasks 7. **Log transitions** - Track job state changes 8. **Retry appropriately** - Exponential backoff for transient errors 9. **Don't retry permanent failures** - Validation errors, invalid credentials 10. **Monitor queue depth** - Alert on backlog growth
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

python-project-structure

Python project organization, module architecture, and public API

coding
⭐1
# Python Project Structure & Module Architecture Design well-organized Python projects with clear module boundaries, explicit public interfaces, and maintainable directory structures. Good organization makes code discoverable and changes predictable. ## When to Use This Skill - Starting a new Python project from scratch - Reorganizing an existing codebase for clarity - Defining module public APIs with `__all__` - Deciding between flat and nested directory structures - Determining test file placement strategies - Creating reusable library packages ## Core Concepts ### 1. Module Cohesion Group related code that changes together. A module should have a single, clear purpose. ### 2. Explicit Interfaces Define what's public with `__all__`. Everything not listed is an internal implementation detail. ### 3. Flat Hierarchies Prefer shallow directory structures. Add depth only for genuine sub-domains. ### 4. Consistent Conventions Apply naming and organization patterns uniformly across the project. ## Quick Start ``` myproject/ β”œβ”€β”€ src/ β”‚ └── myproject/ β”‚ β”œβ”€β”€ __init__.py β”‚ β”œβ”€β”€ services/ β”‚ β”œβ”€β”€ models/ β”‚ └── api/ β”œβ”€β”€ tests/ β”œβ”€β”€ pyproject.toml └── README.md ``` ## Fundamental Patterns ### Pattern 1: One Concept Per File Each file should focus on a single concept or closely related set of functions. Consider splitting when a file: - Handles multiple unrelated responsibilities - Grows beyond 300-500 lines (varies by complexity) - Contains classes that change for different reasons ```python # Good: Focused files # user_service.py - User business logic # user_repository.py - User data access # user_models.py - User data structures # Avoid: Kitchen sink files # user.py - Contains service, repository, models, utilities... ``` ### Pattern 2: Explicit Public APIs with `__all__` Define the public interface for every module. Unlisted members are internal implementation details. ```python # mypackage/services/__init__.py from .user_service import UserService from .order_service import OrderService from .exceptions import ServiceError, ValidationError __all__ = [ "UserService", "OrderService", "ServiceError", "ValidationError", ] # Internal helpers remain private by omission # from .internal_helpers import _validate_input # Not exported ``` ### Pattern 3: Flat Directory Structure Prefer minimal nesting. Deep hierarchies make imports verbose and navigation difficult. ``` # Preferred: Flat structure project/ β”œβ”€β”€ api/ β”‚ β”œβ”€β”€ routes.py β”‚ └── middleware.py β”œβ”€β”€ services/ β”‚ β”œβ”€β”€ user_service.py β”‚ └── order_service.py β”œβ”€β”€ models/ β”‚ β”œβ”€β”€ user.py β”‚ └── order.py └── utils/ └── validation.py # Avoid: Deep nesting project/core/internal/services/impl/user/ ``` Add sub-packages only when there's a genuine sub-domain requiring isolation. ### Pattern 4: Test File Organization Choose one approach and apply it consistently throughout the project. **Option A: Colocated Tests** ``` src/ β”œβ”€β”€ user_service.py β”œβ”€β”€ test_user_service.py β”œβ”€β”€ order_service.py └── test_order_service.py ``` Benefits: Tests live next to the code they verify. Easy to see coverage gaps. **Option B: Parallel Test Directory** ``` src/ β”œβ”€β”€ services/ β”‚ β”œβ”€β”€ user_service.py β”‚ └── order_service.py tests/ β”œβ”€β”€ services/ β”‚ β”œβ”€β”€ test_user_service.py β”‚ └── test_order_service.py ``` Benefits: Clean separation between production and test code. Standard for larger projects. ## Advanced Patterns ### Pattern 5: Package Initialization Use `__init__.py` to provide a clean public interface for package consumers. ```python # mypackage/__init__.py """MyPackage - A library for doing useful things.""" from .core import MainClass, HelperClass from .exceptions import PackageError, ConfigError from .config import Settings __all__ = [ "MainClass", "HelperClass", "PackageError", "ConfigError", "Settings", ] __version__ = "1.0.0" ``` Consumers can then import directly from the package: ```python from mypackage import MainClass, Settings ``` ### Pattern 6: Layered Architecture Organize code by architectural layer for clear separation of concerns. ``` myapp/ β”œβ”€β”€ api/ # HTTP handlers, request/response β”‚ β”œβ”€β”€ routes/ β”‚ └── middleware/ β”œβ”€β”€ services/ # Business logic β”œβ”€β”€ repositories/ # Data access β”œβ”€β”€ models/ # Domain entities β”œβ”€β”€ schemas/ # API schemas (Pydantic) └── config/ # Configuration ``` Each layer should only depend on layers below it, never above. ### Pattern 7: Domain-Driven Structure For complex applications, organize by business domain rather than technical layer. ``` ecommerce/ β”œβ”€β”€ users/ β”‚ β”œβ”€β”€ models.py β”‚ β”œβ”€β”€ services.py β”‚ β”œβ”€β”€ repository.py β”‚ └── api.py β”œβ”€β”€ orders/ β”‚ β”œβ”€β”€ models.py β”‚ β”œβ”€β”€ services.py β”‚ β”œβ”€β”€ repository.py β”‚ └── api.py └── shared/ β”œβ”€β”€ database.py └── exceptions.py ``` ## File and Module Naming ### Conventions - Use `snake_case` for all file and module names: `user_repository.py` - Avoid abbreviations that obscure meaning: `user_repository.py` not `usr_repo.py` - Match class names to file names: `UserService` in `user_service.py` ### Import Style Use absolute imports for clarity and reliability: ```python # Preferred: Absolute imports from myproject.services import UserService from myproject.models import User # Avoid: Relative imports from ..services import UserService from . import models ``` Relative imports can break when modules are moved or reorganized. ## Best Practices Summary 1. **Keep files focused** - One concept per file, consider splitting at 300-500 lines (varies by complexity) 2. **Define `__all__` explicitly** - Make public interfaces clear 3. **Prefer flat structures** - Add depth only for genuine sub-domains 4. **Use absolute imports** - More reliable and clearer 5. **Be consistent** - Apply patterns uniformly across the project 6. **Match names to content** - File names should describe their purpose 7. **Separate concerns** - Keep layers distinct and dependencies flowing one direction 8. **Document your structure** - Include a README explaining the organization
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

python-testing-patterns

Implement comprehensive testing strategies with pytest, fixtures,

coding
⭐1
# Python Testing Patterns Comprehensive guide to implementing robust testing strategies in Python using pytest, fixtures, mocking, parameterization, and test-driven development practices. ## When to Use This Skill - Writing unit tests for Python code - Setting up test suites and test infrastructure - Implementing test-driven development (TDD) - Creating integration tests for APIs and services - Mocking external dependencies and services - Testing async code and concurrent operations - Setting up continuous testing in CI/CD - Implementing property-based testing - Testing database operations - Debugging failing tests ## Core Concepts ### 1. Test Types - **Unit Tests**: Test individual functions/classes in isolation - **Integration Tests**: Test interaction between components - **Functional Tests**: Test complete features end-to-end - **Performance Tests**: Measure speed and resource usage ### 2. Test Structure (AAA Pattern) - **Arrange**: Set up test data and preconditions - **Act**: Execute the code under test - **Assert**: Verify the results ### 3. Test Coverage - Measure what code is exercised by tests - Identify untested code paths - Aim for meaningful coverage, not just high percentages ### 4. Test Isolation - Tests should be independent - No shared state between tests - Each test should clean up after itself ## Quick Start ```python # test_example.py def add(a, b): return a + b def test_add(): """Basic test example.""" result = add(2, 3) assert result == 5 def test_add_negative(): """Test with negative numbers.""" assert add(-1, 1) == 0 # Run with: pytest test_example.py ``` ## Fundamental Patterns ### Pattern 1: Basic pytest Tests ```python # test_calculator.py import pytest class Calculator: """Simple calculator for testing.""" def add(self, a: float, b: float) -> float: return a + b def subtract(self, a: float, b: float) -> float: return a - b def multiply(self, a: float, b: float) -> float: return a * b def divide(self, a: float, b: float) -> float: if b == 0: raise ValueError("Cannot divide by zero") return a / b def test_addition(): """Test addition.""" calc = Calculator() assert calc.add(2, 3) == 5 assert calc.add(-1, 1) == 0 assert calc.add(0, 0) == 0 def test_subtraction(): """Test subtraction.""" calc = Calculator() assert calc.subtract(5, 3) == 2 assert calc.subtract(0, 5) == -5 def test_multiplication(): """Test multiplication.""" calc = Calculator() assert calc.multiply(3, 4) == 12 assert calc.multiply(0, 5) == 0 def test_division(): """Test division.""" calc = Calculator() assert calc.divide(6, 3) == 2 assert calc.divide(5, 2) == 2.5 def test_division_by_zero(): """Test division by zero raises error.""" calc = Calculator() with pytest.raises(ValueError, match="Cannot divide by zero"): calc.divide(5, 0) ``` ### Pattern 2: Fixtures for Setup and Teardown ```python # test_database.py import pytest from typing import Generator class Database: """Simple database class.""" def __init__(self, connection_string: str): self.connection_string = connection_string self.connected = False def connect(self): """Connect to database.""" self.connected = True def disconnect(self): """Disconnect from database.""" self.connected = False def query(self, sql: str) -> list: """Execute query.""" if not self.connected: raise RuntimeError("Not connected") return [{"id": 1, "name": "Test"}] @pytest.fixture def db() -> Generator[Database, None, None]: """Fixture that provides connected database.""" # Setup database = Database("sqlite:///:memory:") database.connect() # Provide to test yield database # Teardown database.disconnect() def test_database_query(db): """Test database query with fixture.""" results = db.query("SELECT * FROM users") assert len(results) == 1 assert results[0]["name"] == "Test" @pytest.fixture(scope="session") def app_config(): """Session-scoped fixture - created once per test session.""" return { "database_url": "postgresql://localhost/test", "api_key": "test-key", "debug": True } @pytest.fixture(scope="module") def api_client(app_config): """Module-scoped fixture - created once per test module.""" # Setup expensive resource client = {"config": app_config, "session": "active"} yield client # Cleanup client["session"] = "closed" def test_api_client(api_client): """Test using api client fixture.""" assert api_client["session"] == "active" assert api_client["config"]["debug"] is True ``` ### Pattern 3: Parameterized Tests ```python # test_validation.py import pytest def is_valid_email(email: str) -> bool: """Check if email is valid.""" return "@" in email and "." in email.split("@")[1] @pytest.mark.parametrize("email,expected", [ ("user@example.com", True), ("test.user@domain.co.uk", True), ("invalid.email", False), ("@example.com", False), ("user@domain", False), ("", False), ]) def test_email_validation(email, expected): """Test email validation with various inputs.""" assert is_valid_email(email) == expected @pytest.mark.parametrize("a,b,expected", [ (2, 3, 5), (0, 0, 0), (-1, 1, 0), (100, 200, 300), (-5, -5, -10), ]) def test_addition_parameterized(a, b, expected): """Test addition with multiple parameter sets.""" from test_calculator import Calculator calc = Calculator() assert calc.add(a, b) == expected # Using pytest.param for special cases @pytest.mark.parametrize("value,expected", [ pytest.param(1, True, id="positive"), pytest.param(0, False, id="zero"), pytest.param(-1, False, id="negative"), ]) def test_is_positive(value, expected): """Test with custom test IDs.""" assert (value > 0) == expected ``` ### Pattern 4: Mocking with unittest.mock ```python # test_api_client.py import pytest from unittest.mock import Mock, patch, MagicMock import requests class APIClient: """Simple API client.""" def __init__(self, base_url: str): self.base_url = base_url def get_user(self, user_id: int) -> dict: """Fetch user from API.""" response = requests.get(f"{self.base_url}/users/{user_id}") response.raise_for_status() return response.json() def create_user(self, data: dict) -> dict: """Create new user.""" response = requests.post(f"{self.base_url}/users", json=data) response.raise_for_status() return response.json() def test_get_user_success(): """Test successful API call with mock.""" client = APIClient("https://api.example.com") mock_response = Mock() mock_response.json.return_value = {"id": 1, "name": "John Doe"} mock_response.raise_for_status.return_value = None with patch("requests.get", return_value=mock_response) as mock_get: user = client.get_user(1) assert user["id"] == 1 assert user["name"] == "John Doe" mock_get.assert_called_once_with("https://api.example.com/users/1") def test_get_user_not_found(): """Test API call with 404 error.""" client = APIClient("https://api.example.com") mock_response = Mock() mock_response.raise_for_status.side_effect = requests.HTTPError("404 Not Found") with patch("requests.get", return_value=mock_response): with pytest.raises(requests.HTTPError): client.get_user(999) @patch("requests.post") def test_create_user(mock_post): """Test user creation with decorator syntax.""" client = APIClient("https://api.example.com") mock_post.return_value.json.return_value = {"id": 2, "name": "Jane Doe"} mock_post.return_value.raise_for_status.return_value = None user_data = {"name": "Jane Doe", "email": "jane@example.com"} result = client.create_user(user_data) assert result["id"] == 2 mock_post.assert_called_once() call_args = mock_post.call_args assert call_args.kwargs["json"] == user_data ``` ### Pattern 5: Testing Exceptions ```python # test_exceptions.py import pytest def divide(a: float, b: float) -> float: """Divide a by b.""" if b == 0: raise ZeroDivisionError("Division by zero") if not isinstance(a, (int, float)) or not isinstance(b, (int, float)): raise TypeError("Arguments must be numbers") return a / b def test_zero_division(): """Test exception is raised for division by zero.""" with pytest.raises(ZeroDivisionError): divide(10, 0) def test_zero_division_with_message(): """Test exception message.""" with pytest.raises(ZeroDivisionError, match="Division by zero"): divide(5, 0) def test_type_error(): """Test type error exception.""" with pytest.raises(TypeError, match="must be numbers"): divide("10", 5) def test_exception_info(): """Test accessing exception info.""" with pytest.raises(ValueError) as exc_info: int("not a number") assert "invalid literal" in str(exc_info.value) ``` ## Advanced Patterns ### Pattern 6: Testing Async Code ```python # test_async.py import pytest import asyncio async def fetch_data(url: str) -> dict: """Fetch data asynchronously.""" await asyncio.sleep(0.1) return {"url": url, "data": "result"} @pytest.mark.asyncio async def test_fetch_data(): """Test async function.""" result = await fetch_data("https://api.example.com") assert result["url"] == "https://api.example.com" assert "data" in result @pytest.mark.asyncio async def test_concurrent_fetches(): """Test concurrent async operations.""" urls = ["url1", "url2", "url3"] tasks = [fetch_data(url) for url in urls] results = await asyncio.gather(*tasks) assert len(results) == 3 assert all("data" in r for r in results) @pytest.fixture async def async_client(): """Async fixture.""" client = {"connected": True} yield client client["connected"] = False @pytest.mark.asyncio async def test_with_async_fixture(async_client): """Test using async fixture.""" assert async_client["connected"] is True ``` ### Pattern 7: Monkeypatch for Testing ```python # test_environment.py import os import pytest def get_database_url() -> str: """Get database URL from environment.""" return os.environ.get("DATABASE_URL", "sqlite:///:memory:") def test_database_url_default(): """Test default database URL.""" # Will use actual environment variable if set url = get_database_url() assert url def test_database_url_custom(monkeypatch): """Test custom database URL with monkeypatch.""" monkeypatch.setenv("DATABASE_URL", "postgresql://localhost/test") assert get_database_url() == "postgresql://localhost/test" def test_database_url_not_set(monkeypatch): """Test when env var is not set.""" monkeypatch.delenv("DATABASE_URL", raising=False) assert get_database_url() == "sqlite:///:memory:" class Config: """Configuration class.""" def __init__(self): self.api_key = "production-key" def get_api_key(self): return self.api_key def test_monkeypatch_attribute(monkeypatch): """Test monkeypatching object attributes.""" config = Config() monkeypatch.setattr(config, "api_key", "test-key") assert config.get_api_key() == "test-key" ``` ### Pattern 8: Temporary Files and Directories ```python # test_file_operations.py import pytest from pathlib import Path def save_data(filepath: Path, data: str): """Save data to file.""" filepath.write_text(data) def load_data(filepath: Path) -> str: """Load data from file.""" return filepath.read_text() def test_file_operations(tmp_path): """Test file operations with temporary directory.""" # tmp_path is a pathlib.Path object test_file = tmp_path / "test_data.txt" # Save data save_data(test_file, "Hello, World!") # Verify file exists assert test_file.exists() # Load and verify data data = load_data(test_file) assert data == "Hello, World!" def test_multiple_files(tmp_path): """Test with multiple temporary files.""" files = { "file1.txt": "Content 1", "file2.txt": "Content 2", "file3.txt": "Content 3" } for filename, content in files.items(): filepath = tmp_path / filename save_data(filepath, content) # Verify all files created assert len(list(tmp_path.iterdir())) == 3 # Verify contents for filename, expected_content in files.items(): filepath = tmp_path / filename assert load_data(filepath) == expected_content ``` ### Pattern 9: Custom Fixtures and Conftest ```python # conftest.py """Shared fixtures for all tests.""" import pytest @pytest.fixture(scope="session") def database_url(): """Provide database URL for all tests.""" return "postgresql://localhost/test_db" @pytest.fixture(autouse=True) def reset_database(database_url): """Auto-use fixture that runs before each test.""" # Setup: Clear database print(f"Clearing database: {database_url}") yield # Teardown: Clean up print("Test completed") @pytest.fixture def sample_user(): """Provide sample user data.""" return { "id": 1, "name": "Test User", "email": "test@example.com" } @pytest.fixture def sample_users(): """Provide list of sample users.""" return [ {"id": 1, "name": "User 1"}, {"id": 2, "name": "User 2"}, {"id": 3, "name": "User 3"}, ] # Parametrized fixture @pytest.fixture(params=["sqlite", "postgresql", "mysql"]) def db_backend(request): """Fixture that runs tests with different database backends.""" return request.param def test_with_db_backend(db_backend): """This test will run 3 times with different backends.""" print(f"Testing with {db_backend}") assert db_backend in ["sqlite", "postgresql", "mysql"] ``` ### Pattern 10: Property-Based Testing ```python # test_properties.py from hypothesis import given, strategies as st import pytest def reverse_string(s: str) -> str: """Reverse a string.""" return s[::-1] @given(st.text()) def test_reverse_twice_is_original(s): """Property: reversing twice returns original.""" assert reverse_string(reverse_string(s)) == s @given(st.text()) def test_reverse_length(s): """Property: reversed string has same length.""" assert len(reverse_string(s)) == len(s) @given(st.integers(), st.integers()) def test_addition_commutative(a, b): """Property: addition is commutative.""" assert a + b == b + a @given(st.lists(st.integers())) def test_sorted_list_properties(lst): """Property: sorted list is ordered.""" sorted_lst = sorted(lst) # Same length assert len(sorted_lst) == len(lst) # All elements present assert set(sorted_lst) == set(lst) # Is ordered for i in range(len(sorted_lst) - 1): assert sorted_lst[i] <= sorted_lst[i + 1] ``` ## Test Design Principles ### One Behavior Per Test Each test should verify exactly one behavior. This makes failures easy to diagnose and tests easy to maintain. ```python # BAD - testing multiple behaviors def test_user_service(): user = service.create_user(data) assert user.id is not None assert user.email == data["email"] updated = service.update_user(user.id, {"name": "New"}) assert updated.name == "New" # GOOD - focused tests def test_create_user_assigns_id(): user = service.create_user(data) assert user.id is not None def test_create_user_stores_email(): user = service.create_user(data) assert user.email == data["email"] def test_update_user_changes_name(): user = service.create_user(data) updated = service.update_user(user.id, {"name": "New"}) assert updated.name == "New" ``` ### Test Error Paths Always test failure cases, not just happy paths. ```python def test_get_user_raises_not_found(): with pytest.raises(UserNotFoundError) as exc_info: service.get_user("nonexistent-id") assert "nonexistent-id" in str(exc_info.value) def test_create_user_rejects_invalid_email(): with pytest.raises(ValueError, match="Invalid email format"): service.create_user({"email": "not-an-email"}) ``` ## Testing Best Practices ### Test Organization ```python # tests/ # __init__.py # conftest.py # Shared fixtures # test_unit/ # Unit tests # test_models.py # test_utils.py # test_integration/ # Integration tests # test_api.py # test_database.py # test_e2e/ # End-to-end tests # test_workflows.py ``` ### Test Naming Convention A common pattern: `test_<unit>_<scenario>_<expected_outcome>`. Adapt to your team's preferences. ```python # Pattern: test_<unit>_<scenario>_<expected> def test_create_user_with_valid_data_returns_user(): ... def test_create_user_with_duplicate_email_raises_conflict(): ... def test_get_user_with_unknown_id_returns_none(): ... # Good test names - clear and descriptive def test_user_creation_with_valid_data(): """Clear name describes what is being tested.""" pass def test_login_fails_with_invalid_password(): """Name describes expected behavior.""" pass def test_api_returns_404_for_missing_resource(): """Specific about inputs and expected outcomes.""" pass # Bad test names - avoid these def test_1(): # Not descriptive pass def test_user(): # Too vague pass def test_function(): # Doesn't explain what's tested pass ``` ### Testing Retry Behavior Verify that retry logic works correctly using mock side effects. ```python from unittest.mock import Mock def test_retries_on_transient_error(): """Test that service retries on transient failures.""" client = Mock() # Fail twice, then succeed client.request.side_effect = [ ConnectionError("Failed"), ConnectionError("Failed"), {"status": "ok"}, ] service = ServiceWithRetry(client, max_retries=3) result = service.fetch() assert result == {"status": "ok"} assert client.request.call_count == 3 def test_gives_up_after_max_retries(): """Test that service stops retrying after max attempts.""" client = Mock() client.request.side_effect = ConnectionError("Failed") service = ServiceWithRetry(client, max_retries=3) with pytest.raises(ConnectionError): service.fetch() assert client.request.call_count == 3 def test_does_not_retry_on_permanent_error(): """Test that permanent errors are not retried.""" client = Mock() client.request.side_effect = ValueError("Invalid input") service = ServiceWithRetry(client, max_retries=3) with pytest.raises(ValueError): service.fetch() # Only called once - no retry for ValueError assert client.request.call_count == 1 ``` ### Mocking Time with Freezegun Use freezegun
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

backtesting-frameworks

Build robust backtesting systems for trading strategies with proper

coding
⭐1
# Backtesting Frameworks Build robust, production-grade backtesting systems that avoid common pitfalls and produce reliable strategy performance estimates. ## When to Use This Skill - Developing trading strategy backtests - Building backtesting infrastructure - Validating strategy performance - Avoiding common backtesting biases - Implementing walk-forward analysis - Comparing strategy alternatives ## Core Concepts ### 1. Backtesting Biases | Bias | Description | Mitigation | | ---------------- | ------------------------- | ----------------------- | | **Look-ahead** | Using future information | Point-in-time data | | **Survivorship** | Only testing on survivors | Use delisted securities | | **Overfitting** | Curve-fitting to history | Out-of-sample testing | | **Selection** | Cherry-picking strategies | Pre-registration | | **Transaction** | Ignoring trading costs | Realistic cost models | ### 2. Proper Backtest Structure ``` Historical Data β”‚ β–Ό β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Training Set β”‚ β”‚ (Strategy Development & Optimization) β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β–Ό β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Validation Set β”‚ β”‚ (Parameter Selection, No Peeking) β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β–Ό β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Test Set β”‚ β”‚ (Final Performance Evaluation) β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` ### 3. Walk-Forward Analysis ``` Window 1: [Train──────][Test] Window 2: [Train──────][Test] Window 3: [Train──────][Test] Window 4: [Train──────][Test] ─────▢ Time ``` ## Implementation Patterns ### Pattern 1: Event-Driven Backtester ```python from abc import ABC, abstractmethod from dataclasses import dataclass, field from datetime import datetime from decimal import Decimal from enum import Enum from typing import Dict, List, Optional import pandas as pd import numpy as np class OrderSide(Enum): BUY = "buy" SELL = "sell" class OrderType(Enum): MARKET = "market" LIMIT = "limit" STOP = "stop" @dataclass class Order: symbol: str side: OrderSide quantity: Decimal order_type: OrderType limit_price: Optional[Decimal] = None stop_price: Optional[Decimal] = None timestamp: Optional[datetime] = None @dataclass class Fill: order: Order fill_price: Decimal fill_quantity: Decimal commission: Decimal slippage: Decimal timestamp: datetime @dataclass class Position: symbol: str quantity: Decimal = Decimal("0") avg_cost: Decimal = Decimal("0") realized_pnl: Decimal = Decimal("0") def update(self, fill: Fill) -> None: if fill.order.side == OrderSide.BUY: new_quantity = self.quantity + fill.fill_quantity if new_quantity != 0: self.avg_cost = ( (self.quantity * self.avg_cost + fill.fill_quantity * fill.fill_price) / new_quantity ) self.quantity = new_quantity else: self.realized_pnl += fill.fill_quantity * (fill.fill_price - self.avg_cost) self.quantity -= fill.fill_quantity @dataclass class Portfolio: cash: Decimal positions: Dict[str, Position] = field(default_factory=dict) def get_position(self, symbol: str) -> Position: if symbol not in self.positions: self.positions[symbol] = Position(symbol=symbol) return self.positions[symbol] def process_fill(self, fill: Fill) -> None: position = self.get_position(fill.order.symbol) position.update(fill) if fill.order.side == OrderSide.BUY: self.cash -= fill.fill_price * fill.fill_quantity + fill.commission else: self.cash += fill.fill_price * fill.fill_quantity - fill.commission def get_equity(self, prices: Dict[str, Decimal]) -> Decimal: equity = self.cash for symbol, position in self.positions.items(): if position.quantity != 0 and symbol in prices: equity += position.quantity * prices[symbol] return equity class Strategy(ABC): @abstractmethod def on_bar(self, timestamp: datetime, data: pd.DataFrame) -> List[Order]: pass @abstractmethod def on_fill(self, fill: Fill) -> None: pass class ExecutionModel(ABC): @abstractmethod def execute(self, order: Order, bar: pd.Series) -> Optional[Fill]: pass class SimpleExecutionModel(ExecutionModel): def __init__(self, slippage_bps: float = 10, commission_per_share: float = 0.01): self.slippage_bps = slippage_bps self.commission_per_share = commission_per_share def execute(self, order: Order, bar: pd.Series) -> Optional[Fill]: if order.order_type == OrderType.MARKET: base_price = Decimal(str(bar["open"])) # Apply slippage slippage_mult = 1 + (self.slippage_bps / 10000) if order.side == OrderSide.BUY: fill_price = base_price * Decimal(str(slippage_mult)) else: fill_price = base_price / Decimal(str(slippage_mult)) commission = order.quantity * Decimal(str(self.commission_per_share)) slippage = abs(fill_price - base_price) * order.quantity return Fill( order=order, fill_price=fill_price, fill_quantity=order.quantity, commission=commission, slippage=slippage, timestamp=bar.name ) return None class Backtester: def __init__( self, strategy: Strategy, execution_model: ExecutionModel, initial_capital: Decimal = Decimal("100000") ): self.strategy = strategy self.execution_model = execution_model self.portfolio = Portfolio(cash=initial_capital) self.equity_curve: List[tuple] = [] self.trades: List[Fill] = [] def run(self, data: pd.DataFrame) -> pd.DataFrame: """Run backtest on OHLCV data with DatetimeIndex.""" pending_orders: List[Order] = [] for timestamp, bar in data.iterrows(): # Execute pending orders at today's prices for order in pending_orders: fill = self.execution_model.execute(order, bar) if fill: self.portfolio.process_fill(fill) self.strategy.on_fill(fill) self.trades.append(fill) pending_orders.clear() # Get current prices for equity calculation prices = {data.index.name or "default": Decimal(str(bar["close"]))} equity = self.portfolio.get_equity(prices) self.equity_curve.append((timestamp, float(equity))) # Generate new orders for next bar new_orders = self.strategy.on_bar(timestamp, data.loc[:timestamp]) pending_orders.extend(new_orders) return self._create_results() def _create_results(self) -> pd.DataFrame: equity_df = pd.DataFrame(self.equity_curve, columns=["timestamp", "equity"]) equity_df.set_index("timestamp", inplace=True) equity_df["returns"] = equity_df["equity"].pct_change() return equity_df ``` ### Pattern 2: Vectorized Backtester (Fast) ```python import pandas as pd import numpy as np from typing import Callable, Dict, Any class VectorizedBacktester: """Fast vectorized backtester for simple strategies.""" def __init__( self, initial_capital: float = 100000, commission: float = 0.001, # 0.1% slippage: float = 0.0005 # 0.05% ): self.initial_capital = initial_capital self.commission = commission self.slippage = slippage def run( self, prices: pd.DataFrame, signal_func: Callable[[pd.DataFrame], pd.Series] ) -> Dict[str, Any]: """ Run backtest with signal function. Args: prices: DataFrame with 'close' column signal_func: Function that returns position signals (-1, 0, 1) Returns: Dictionary with results """ # Generate signals (shifted to avoid look-ahead) signals = signal_func(prices).shift(1).fillna(0) # Calculate returns returns = prices["close"].pct_change() # Calculate strategy returns with costs position_changes = signals.diff().abs() trading_costs = position_changes * (self.commission + self.slippage) strategy_returns = signals * returns - trading_costs # Build equity curve equity = (1 + strategy_returns).cumprod() * self.initial_capital # Calculate metrics results = { "equity": equity, "returns": strategy_returns, "signals": signals, "metrics": self._calculate_metrics(strategy_returns, equity) } return results def _calculate_metrics( self, returns: pd.Series, equity: pd.Series ) -> Dict[str, float]: """Calculate performance metrics.""" total_return = (equity.iloc[-1] / self.initial_capital) - 1 annual_return = (1 + total_return) ** (252 / len(returns)) - 1 annual_vol = returns.std() * np.sqrt(252) sharpe = annual_return / annual_vol if annual_vol > 0 else 0 # Drawdown rolling_max = equity.cummax() drawdown = (equity - rolling_max) / rolling_max max_drawdown = drawdown.min() # Win rate winning_days = (returns > 0).sum() total_days = (returns != 0).sum() win_rate = winning_days / total_days if total_days > 0 else 0 return { "total_return": total_return, "annual_return": annual_return, "annual_volatility": annual_vol, "sharpe_ratio": sharpe, "max_drawdown": max_drawdown, "win_rate": win_rate, "num_trades": int((returns != 0).sum()) } # Example usage def momentum_signal(prices: pd.DataFrame, lookback: int = 20) -> pd.Series: """Simple momentum strategy: long when price > SMA, else flat.""" sma = prices["close"].rolling(lookback).mean() return (prices["close"] > sma).astype(int) # Run backtest # backtester = VectorizedBacktester() # results = backtester.run(price_data, lambda p: momentum_signal(p, 50)) ``` ### Pattern 3: Walk-Forward Optimization ```python from typing import Callable, Dict, List, Tuple, Any import pandas as pd import numpy as np from itertools import product class WalkForwardOptimizer: """Walk-forward analysis with anchored or rolling windows.""" def __init__( self, train_period: int, test_period: int, anchored: bool = False, n_splits: int = None ): """ Args: train_period: Number of bars in training window test_period: Number of bars in test window anchored: If True, training always starts from beginning n_splits: Number of train/test splits (auto-calculated if None) """ self.train_period = train_period self.test_period = test_period self.anchored = anchored self.n_splits = n_splits def generate_splits( self, data: pd.DataFrame ) -> List[Tuple[pd.DataFrame, pd.DataFrame]]: """Generate train/test splits.""" splits = [] n = len(data) if self.n_splits: step = (n - self.train_period) // self.n_splits else: step = self.test_period start = 0 while start + self.train_period + self.test_period <= n: if self.anchored: train_start = 0 else: train_start = start train_end = start + self.train_period test_end = min(train_end + self.test_period, n) train_data = data.iloc[train_start:train_end] test_data = data.iloc[train_end:test_end] splits.append((train_data, test_data)) start += step return splits def optimize( self, data: pd.DataFrame, strategy_func: Callable, param_grid: Dict[str, List], metric: str = "sharpe_ratio" ) -> Dict[str, Any]: """ Run walk-forward optimization. Args: data: Full dataset strategy_func: Function(data, **params) -> results dict param_grid: Parameter combinations to test metric: Metric to optimize Returns: Combined results from all test periods """ splits = self.generate_splits(data) all_results = [] optimal_params_history = [] for i, (train_data, test_data) in enumerate(splits): # Optimize on training data best_params, best_metric = self._grid_search( train_data, strategy_func, param_grid, metric ) optimal_params_history.append(best_params) # Test with optimal params test_results = strategy_func(test_data, **best_params) test_results["split"] = i test_results["params"] = best_params all_results.append(test_results) print(f"Split {i+1}/{len(splits)}: " f"Best {metric}={best_metric:.4f}, params={best_params}") return { "split_results": all_results, "param_history": optimal_params_history, "combined_equity": self._combine_equity_curves(all_results) } def _grid_search( self, data: pd.DataFrame, strategy_func: Callable, param_grid: Dict[str, List], metric: str ) -> Tuple[Dict, float]: """Grid search for best parameters.""" best_params = None best_metric = -np.inf # Generate all parameter combinations param_names = list(param_grid.keys()) param_values = list(param_grid.values()) for values in product(*param_values): params = dict(zip(param_names, values)) results = strategy_func(data, **params) if results["metrics"][metric] > best_metric: best_metric = results["metrics"][metric] best_params = params return best_params, best_metric def _combine_equity_curves( self, results: List[Dict] ) -> pd.Series: """Combine equity curves from all test periods.""" combined = pd.concat([r["equity"] for r in results]) return combined ``` ### Pattern 4: Monte Carlo Analysis ```python import numpy as np import pandas as pd from typing import Dict, List class MonteCarloAnalyzer: """Monte Carlo simulation for strategy robustness.""" def __init__(self, n_simulations: int = 1000, confidence: float = 0.95): self.n_simulations = n_simulations self.confidence = confidence def bootstrap_returns( self, returns: pd.Series, n_periods: int = None ) -> np.ndarray: """ Bootstrap simulation by resampling returns. Args: returns: Historical returns series n_periods: Length of each simulation (default: same as input) Returns: Array of shape (n_simulations, n_periods) """ if n_periods is None: n_periods = len(returns) simulations = np.zeros((self.n_simulations, n_periods)) for i in range(self.n_simulations): # Resample with replacement simulated_returns = np.random.choice( returns.values, size=n_periods, replace=True ) simulations[i] = simulated_returns return simulations def analyze_drawdowns( self, returns: pd.Series ) -> Dict[str, float]: """Analyze drawdown distribution via simulation.""" simulations = self.bootstrap_returns(returns) max_drawdowns = [] for sim_returns in simulations: equity = (1 + sim_returns).cumprod() rolling_max = np.maximum.accumulate(equity) drawdowns = (equity - rolling_max) / rolling_max max_drawdowns.append(drawdowns.min()) max_drawdowns = np.array(max_drawdowns) return { "expected_max_dd": np.mean(max_drawdowns), "median_max_dd": np.median(max_drawdowns), f"worst_{int(self.confidence*100)}pct": np.percentile( max_drawdowns, (1 - self.confidence) * 100 ), "worst_case": max_drawdowns.min() } def probability_of_loss( self, returns: pd.Series, holding_periods: List[int] = [21, 63, 126, 252] ) -> Dict[int, float]: """Calculate probability of loss over various holding periods.""" results = {} for period in holding_periods: if period > len(returns): continue simulations = self.bootstrap_returns(returns, period) total_returns = (1 + simulations).prod(axis=1) - 1 prob_loss = (total_returns < 0).mean() results[period] = prob_loss return results def confidence_interval( self, returns: pd.Series, periods: int = 252 ) -> Dict[str, float]: """Calculate confidence interval for future returns.""" simulations = self.bootstrap_returns(returns, periods) total_returns = (1 + simulations).prod(axis=1) - 1 lower = (1 - self.confidence) / 2 upper = 1 - lower return { "expected": total_returns.mean(), "lower_bound": np.percentile(total_returns, lower * 100), "upper_bound": np.percentile(total_returns, upper * 100), "std": total_returns.std() } ``` ## Performance Metrics ```python def calculate_metrics(returns: pd.Series, rf_rate: float = 0.02) -> Dict[str, float]: """Calculate comprehensive performance metrics.""" # Annualization factor (assuming daily returns) ann_factor = 252 # Basic metrics total_return = (1 + returns).prod() - 1 annual_return = (1 + total_return) ** (ann_factor / len(returns)) - 1 annual_vol = returns.std() * np.sqrt(ann_factor) # Risk-adjusted returns sharpe = (annual_return - rf_rate) / annual_vol if annual_vol > 0 else 0 # Sortino (downside deviation) downside_returns = returns[returns < 0] downside_vol = downside_returns.std() * np.sqrt(ann_factor) sortino = (annual_return - rf_rate) / downside_vol if downside_vol > 0 else 0 # Calmar ratio equity = (1 + returns).cumprod() rolling_max = equity.cummax() drawdowns = (equity - rolling_max) / rolling_max max_drawdown = drawdowns.min() calmar = annual_return / abs(max_drawdown) if max_drawdown != 0 else 0 # Win rate and profit factor wins = returns[returns > 0] losses = return
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

bats-testing-patterns

Master Bash Automated Testing System (Bats) for comprehensive shell

coding
⭐1
# Bats Testing Patterns Comprehensive guidance for writing comprehensive unit tests for shell scripts using Bats (Bash Automated Testing System), including test patterns, fixtures, and best practices for production-grade shell testing. ## When to Use This Skill - Writing unit tests for shell scripts - Implementing test-driven development (TDD) for scripts - Setting up automated testing in CI/CD pipelines - Testing edge cases and error conditions - Validating behavior across different shell environments - Building maintainable test suites for scripts - Creating fixtures for complex test scenarios - Testing multiple shell dialects (bash, sh, dash) ## Bats Fundamentals ### What is Bats? Bats (Bash Automated Testing System) is a TAP (Test Anything Protocol) compliant testing framework for shell scripts that provides: - Simple, natural test syntax - TAP output format compatible with CI systems - Fixtures and setup/teardown support - Assertion helpers - Parallel test execution ### Installation ```bash # macOS with Homebrew brew install bats-core # Ubuntu/Debian git clone https://github.com/bats-core/bats-core.git cd bats-core ./install.sh /usr/local # From npm (Node.js) npm install --global bats # Verify installation bats --version ``` ### File Structure ``` project/ β”œβ”€β”€ bin/ β”‚ β”œβ”€β”€ script.sh β”‚ └── helper.sh β”œβ”€β”€ tests/ β”‚ β”œβ”€β”€ test_script.bats β”‚ β”œβ”€β”€ test_helper.sh β”‚ β”œβ”€β”€ fixtures/ β”‚ β”‚ β”œβ”€β”€ input.txt β”‚ β”‚ └── expected_output.txt β”‚ └── helpers/ β”‚ └── mocks.bash └── README.md ``` ## Basic Test Structure ### Simple Test File ```bash #!/usr/bin/env bats # Load test helper if present load test_helper # Setup runs before each test setup() { export TMPDIR=$(mktemp -d) } # Teardown runs after each test teardown() { rm -rf "$TMPDIR" } # Test: simple assertion @test "Function returns 0 on success" { run my_function "input" [ "$status" -eq 0 ] } # Test: output verification @test "Function outputs correct result" { run my_function "test" [ "$output" = "expected output" ] } # Test: error handling @test "Function returns 1 on missing argument" { run my_function [ "$status" -eq 1 ] } ``` ## Assertion Patterns ### Exit Code Assertions ```bash #!/usr/bin/env bats @test "Command succeeds" { run true [ "$status" -eq 0 ] } @test "Command fails as expected" { run false [ "$status" -ne 0 ] } @test "Command returns specific exit code" { run my_function --invalid [ "$status" -eq 127 ] } @test "Can capture command result" { run echo "hello" [ $status -eq 0 ] [ "$output" = "hello" ] } ``` ### Output Assertions ```bash #!/usr/bin/env bats @test "Output matches string" { result=$(echo "hello world") [ "$result" = "hello world" ] } @test "Output contains substring" { result=$(echo "hello world") [[ "$result" == *"world"* ]] } @test "Output matches pattern" { result=$(date +%Y) [[ "$result" =~ ^[0-9]{4}$ ]] } @test "Multi-line output" { run printf "line1\nline2\nline3" [ "$output" = "line1 line2 line3" ] } @test "Lines variable contains output" { run printf "line1\nline2\nline3" [ "${lines[0]}" = "line1" ] [ "${lines[1]}" = "line2" ] [ "${lines[2]}" = "line3" ] } ``` ### File Assertions ```bash #!/usr/bin/env bats @test "File is created" { [ ! -f "$TMPDIR/output.txt" ] my_function > "$TMPDIR/output.txt" [ -f "$TMPDIR/output.txt" ] } @test "File contents match expected" { my_function > "$TMPDIR/output.txt" [ "$(cat "$TMPDIR/output.txt")" = "expected content" ] } @test "File is readable" { touch "$TMPDIR/test.txt" [ -r "$TMPDIR/test.txt" ] } @test "File has correct permissions" { touch "$TMPDIR/test.txt" chmod 644 "$TMPDIR/test.txt" [ "$(stat -f %OLp "$TMPDIR/test.txt")" = "644" ] } @test "File size is correct" { echo -n "12345" > "$TMPDIR/test.txt" [ "$(wc -c < "$TMPDIR/test.txt")" -eq 5 ] } ``` ## Setup and Teardown Patterns ### Basic Setup and Teardown ```bash #!/usr/bin/env bats setup() { # Create test directory TEST_DIR=$(mktemp -d) export TEST_DIR # Source script under test source "${BATS_TEST_DIRNAME}/../bin/script.sh" } teardown() { # Clean up temporary directory rm -rf "$TEST_DIR" } @test "Test using TEST_DIR" { touch "$TEST_DIR/file.txt" [ -f "$TEST_DIR/file.txt" ] } ``` ### Setup with Resources ```bash #!/usr/bin/env bats setup() { # Create directory structure mkdir -p "$TMPDIR/data/input" mkdir -p "$TMPDIR/data/output" # Create test fixtures echo "line1" > "$TMPDIR/data/input/file1.txt" echo "line2" > "$TMPDIR/data/input/file2.txt" # Initialize environment export DATA_DIR="$TMPDIR/data" export INPUT_DIR="$DATA_DIR/input" export OUTPUT_DIR="$DATA_DIR/output" } teardown() { rm -rf "$TMPDIR/data" } @test "Processes input files" { run my_process_script "$INPUT_DIR" "$OUTPUT_DIR" [ "$status" -eq 0 ] [ -f "$OUTPUT_DIR/file1.txt" ] } ``` ### Global Setup/Teardown ```bash #!/usr/bin/env bats # Load shared setup from test_helper.sh load test_helper # setup_file runs once before all tests setup_file() { export SHARED_RESOURCE=$(mktemp -d) echo "Expensive setup" > "$SHARED_RESOURCE/data.txt" } # teardown_file runs once after all tests teardown_file() { rm -rf "$SHARED_RESOURCE" } @test "First test uses shared resource" { [ -f "$SHARED_RESOURCE/data.txt" ] } @test "Second test uses shared resource" { [ -d "$SHARED_RESOURCE" ] } ``` ## Mocking and Stubbing Patterns ### Function Mocking ```bash #!/usr/bin/env bats # Mock external command my_external_tool() { echo "mocked output" return 0 } @test "Function uses mocked tool" { export -f my_external_tool run my_function [[ "$output" == *"mocked output"* ]] } ``` ### Command Stubbing ```bash #!/usr/bin/env bats setup() { # Create stub directory STUBS_DIR="$TMPDIR/stubs" mkdir -p "$STUBS_DIR" # Add to PATH export PATH="$STUBS_DIR:$PATH" } create_stub() { local cmd="$1" local output="$2" local code="${3:-0}" cat > "$STUBS_DIR/$cmd" <<EOF #!/bin/bash echo "$output" exit $code EOF chmod +x "$STUBS_DIR/$cmd" } @test "Function works with stubbed curl" { create_stub curl "{ \"status\": \"ok\" }" 0 run my_api_function [ "$status" -eq 0 ] } ``` ### Variable Stubbing ```bash #!/usr/bin/env bats @test "Function handles environment override" { export MY_SETTING="override_value" run my_function [ "$status" -eq 0 ] [[ "$output" == *"override_value"* ]] } @test "Function uses default when var unset" { unset MY_SETTING run my_function [ "$status" -eq 0 ] [[ "$output" == *"default"* ]] } ``` ## Fixture Management ### Using Fixture Files ```bash #!/usr/bin/env bats # Fixture directory: tests/fixtures/ setup() { FIXTURES_DIR="${BATS_TEST_DIRNAME}/fixtures" WORK_DIR=$(mktemp -d) export WORK_DIR } teardown() { rm -rf "$WORK_DIR" } @test "Process fixture file" { # Copy fixture to work directory cp "$FIXTURES_DIR/input.txt" "$WORK_DIR/input.txt" # Run function run my_process_function "$WORK_DIR/input.txt" # Compare output diff "$WORK_DIR/output.txt" "$FIXTURES_DIR/expected_output.txt" } ``` ### Dynamic Fixture Generation ```bash #!/usr/bin/env bats generate_fixture() { local lines="$1" local file="$2" for i in $(seq 1 "$lines"); do echo "Line $i content" >> "$file" done } @test "Handle large input file" { generate_fixture 1000 "$TMPDIR/large.txt" run my_function "$TMPDIR/large.txt" [ "$status" -eq 0 ] [ "$(wc -l < "$TMPDIR/large.txt")" -eq 1000 ] } ``` ## Advanced Patterns ### Testing Error Conditions ```bash #!/usr/bin/env bats @test "Function fails with missing file" { run my_function "/nonexistent/file.txt" [ "$status" -ne 0 ] [[ "$output" == *"not found"* ]] } @test "Function fails with invalid input" { run my_function "" [ "$status" -ne 0 ] } @test "Function fails with permission denied" { touch "$TMPDIR/readonly.txt" chmod 000 "$TMPDIR/readonly.txt" run my_function "$TMPDIR/readonly.txt" [ "$status" -ne 0 ] chmod 644 "$TMPDIR/readonly.txt" # Cleanup } @test "Function provides helpful error message" { run my_function --invalid-option [ "$status" -ne 0 ] [[ "$output" == *"Usage:"* ]] } ``` ### Testing with Dependencies ```bash #!/usr/bin/env bats setup() { # Check for required tools if ! command -v jq &>/dev/null; then skip "jq is not installed" fi export SCRIPT="${BATS_TEST_DIRNAME}/../bin/script.sh" } @test "JSON parsing works" { skip_if ! command -v jq &>/dev/null run my_json_parser '{"key": "value"}' [ "$status" -eq 0 ] } ``` ### Testing Shell Compatibility ```bash #!/usr/bin/env bats @test "Script works in bash" { bash "${BATS_TEST_DIRNAME}/../bin/script.sh" arg1 } @test "Script works in sh (POSIX)" { sh "${BATS_TEST_DIRNAME}/../bin/script.sh" arg1 } @test "Script works in dash" { if command -v dash &>/dev/null; then dash "${BATS_TEST_DIRNAME}/../bin/script.sh" arg1 else skip "dash not installed" fi } ``` ### Parallel Execution ```bash #!/usr/bin/env bats @test "Multiple independent operations" { run bash -c 'for i in {1..10}; do my_operation "$i" & done wait' [ "$status" -eq 0 ] } @test "Concurrent file operations" { for i in {1..5}; do my_function "$TMPDIR/file$i" & done wait [ -f "$TMPDIR/file1" ] [ -f "$TMPDIR/file5" ] } ``` ## Test Helper Pattern ### test_helper.sh ```bash #!/usr/bin/env bash # Source script under test export SCRIPT_DIR="${BATS_TEST_DIRNAME%/*}/bin" # Common test utilities assert_file_exists() { if [ ! -f "$1" ]; then echo "Expected file to exist: $1" return 1 fi } assert_file_equals() { local file="$1" local expected="$2" if [ ! -f "$file" ]; then echo "File does not exist: $file" return 1 fi local actual=$(cat "$file") if [ "$actual" != "$expected" ]; then echo "File contents do not match" echo "Expected: $expected" echo "Actual: $actual" return 1 fi } # Create temporary test directory setup_test_dir() { export TEST_DIR=$(mktemp -d) } cleanup_test_dir() { rm -rf "$TEST_DIR" } ``` ## Integration with CI/CD ### GitHub Actions Workflow ```yaml name: Tests on: [push, pull_request] jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Install Bats run: | npm install --global bats - name: Run Tests run: | bats tests/*.bats - name: Run Tests with Tap Reporter run: | bats tests/*.bats --tap | tee test_output.tap ``` ### Makefile Integration ```makefile .PHONY: test test-verbose test-tap test: bats tests/*.bats test-verbose: bats tests/*.bats --verbose test-tap: bats tests/*.bats --tap test-parallel: bats tests/*.bats --parallel 4 coverage: test # Optional: Generate coverage reports ``` ## Best Practices 1. **Test one thing per test** - Single responsibility principle 2. **Use descriptive test names** - Clearly states what is being tested 3. **Clean up after tests** - Always remove temporary files in teardown 4. **Test both success and failure paths** - Don't just test happy path 5. **Mock external dependencies** - Isolate unit under test 6. **Use fixtures for complex data** - Makes tests more readable 7. **Run tests in CI/CD** - Catch regressions early 8. **Test across shell dialects** - Ensure portability 9. **Keep tests fast** - Run in parallel when possible 10. **Document complex test setup** - Explain unusual patterns ## Resources - **Bats GitHub**: https://github.com/bats-core/bats-core - **Bats Documentation**: https://bats-core.readthedocs.io/ - **TAP Protocol**: https://testanything.org/ - **Test-Driven Development**: https://en.wikipedia.org/wiki/Test-driven_development
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

react-native-design

Master React Native styling, navigation, and Reanimated animations

coding
⭐1
# React Native Design Master React Native styling patterns, React Navigation, and Reanimated 3 to build performant, cross-platform mobile applications with native-quality user experiences. ## When to Use This Skill - Building cross-platform mobile apps with React Native - Implementing navigation with React Navigation 6+ - Creating performant animations with Reanimated 3 - Styling components with StyleSheet and styled-components - Building responsive layouts for different screen sizes - Implementing platform-specific designs (iOS/Android) - Creating gesture-driven interactions with Gesture Handler - Optimizing React Native performance ## Core Concepts ### 1. StyleSheet and Styling **Basic StyleSheet:** ```typescript import { StyleSheet, View, Text } from 'react-native'; const styles = StyleSheet.create({ container: { flex: 1, padding: 16, backgroundColor: '#ffffff', }, title: { fontSize: 24, fontWeight: '600', color: '#1a1a1a', marginBottom: 8, }, subtitle: { fontSize: 16, color: '#666666', lineHeight: 24, }, }); function Card() { return ( <View style={styles.container}> <Text style={styles.title}>Title</Text> <Text style={styles.subtitle}>Subtitle text</Text> </View> ); } ``` **Dynamic Styles:** ```typescript interface CardProps { variant: 'primary' | 'secondary'; disabled?: boolean; } function Card({ variant, disabled }: CardProps) { return ( <View style={[ styles.card, variant === 'primary' ? styles.primary : styles.secondary, disabled && styles.disabled, ]}> <Text style={styles.text}>Content</Text> </View> ); } const styles = StyleSheet.create({ card: { padding: 16, borderRadius: 12, }, primary: { backgroundColor: '#6366f1', }, secondary: { backgroundColor: '#f3f4f6', borderWidth: 1, borderColor: '#e5e7eb', }, disabled: { opacity: 0.5, }, text: { fontSize: 16, }, }); ``` ### 2. Flexbox Layout **Row and Column Layouts:** ```typescript const styles = StyleSheet.create({ // Vertical stack (column) column: { flexDirection: "column", gap: 12, }, // Horizontal stack (row) row: { flexDirection: "row", alignItems: "center", gap: 8, }, // Space between items spaceBetween: { flexDirection: "row", justifyContent: "space-between", alignItems: "center", }, // Centered content centered: { flex: 1, justifyContent: "center", alignItems: "center", }, // Fill remaining space fill: { flex: 1, }, }); ``` ### 3. React Navigation Setup **Stack Navigator:** ```typescript import { NavigationContainer } from '@react-navigation/native'; import { createNativeStackNavigator } from '@react-navigation/native-stack'; type RootStackParamList = { Home: undefined; Detail: { itemId: string }; Settings: undefined; }; const Stack = createNativeStackNavigator<RootStackParamList>(); function AppNavigator() { return ( <NavigationContainer> <Stack.Navigator initialRouteName="Home" screenOptions={{ headerStyle: { backgroundColor: '#6366f1' }, headerTintColor: '#ffffff', headerTitleStyle: { fontWeight: '600' }, }} > <Stack.Screen name="Home" component={HomeScreen} options={{ title: 'Home' }} /> <Stack.Screen name="Detail" component={DetailScreen} options={({ route }) => ({ title: `Item ${route.params.itemId}`, })} /> <Stack.Screen name="Settings" component={SettingsScreen} /> </Stack.Navigator> </NavigationContainer> ); } ``` **Tab Navigator:** ```typescript import { createBottomTabNavigator } from '@react-navigation/bottom-tabs'; import { Ionicons } from '@expo/vector-icons'; type TabParamList = { Home: undefined; Search: undefined; Profile: undefined; }; const Tab = createBottomTabNavigator<TabParamList>(); function TabNavigator() { return ( <Tab.Navigator screenOptions={({ route }) => ({ tabBarIcon: ({ focused, color, size }) => { const icons: Record<string, keyof typeof Ionicons.glyphMap> = { Home: focused ? 'home' : 'home-outline', Search: focused ? 'search' : 'search-outline', Profile: focused ? 'person' : 'person-outline', }; return <Ionicons name={icons[route.name]} size={size} color={color} />; }, tabBarActiveTintColor: '#6366f1', tabBarInactiveTintColor: '#9ca3af', })} > <Tab.Screen name="Home" component={HomeScreen} /> <Tab.Screen name="Search" component={SearchScreen} /> <Tab.Screen name="Profile" component={ProfileScreen} /> </Tab.Navigator> ); } ``` ### 4. Reanimated 3 Basics **Animated Values:** ```typescript import Animated, { useSharedValue, useAnimatedStyle, withSpring, withTiming, } from 'react-native-reanimated'; function AnimatedBox() { const scale = useSharedValue(1); const opacity = useSharedValue(1); const animatedStyle = useAnimatedStyle(() => ({ transform: [{ scale: scale.value }], opacity: opacity.value, })); const handlePress = () => { scale.value = withSpring(1.2, {}, () => { scale.value = withSpring(1); }); }; return ( <Pressable onPress={handlePress}> <Animated.View style={[styles.box, animatedStyle]} /> </Pressable> ); } ``` **Gesture Handler Integration:** ```typescript import { Gesture, GestureDetector } from 'react-native-gesture-handler'; import Animated, { useSharedValue, useAnimatedStyle, withSpring, } from 'react-native-reanimated'; function DraggableCard() { const translateX = useSharedValue(0); const translateY = useSharedValue(0); const gesture = Gesture.Pan() .onUpdate((event) => { translateX.value = event.translationX; translateY.value = event.translationY; }) .onEnd(() => { translateX.value = withSpring(0); translateY.value = withSpring(0); }); const animatedStyle = useAnimatedStyle(() => ({ transform: [ { translateX: translateX.value }, { translateY: translateY.value }, ], })); return ( <GestureDetector gesture={gesture}> <Animated.View style={[styles.card, animatedStyle]}> <Text>Drag me!</Text> </Animated.View> </GestureDetector> ); } ``` ### 5. Platform-Specific Styling ```typescript import { Platform, StyleSheet } from "react-native"; const styles = StyleSheet.create({ container: { padding: 16, ...Platform.select({ ios: { shadowColor: "#000", shadowOffset: { width: 0, height: 2 }, shadowOpacity: 0.1, shadowRadius: 4, }, android: { elevation: 4, }, }), }, text: { fontFamily: Platform.OS === "ios" ? "SF Pro Text" : "Roboto", fontSize: 16, }, }); // Platform-specific components import { Platform } from "react-native"; const StatusBarHeight = Platform.OS === "ios" ? 44 : 0; ``` ## Quick Start Component ```typescript import React from 'react'; import { View, Text, StyleSheet, Pressable, Image, } from 'react-native'; import Animated, { useSharedValue, useAnimatedStyle, withSpring, } from 'react-native-reanimated'; interface ItemCardProps { title: string; subtitle: string; imageUrl: string; onPress: () => void; } const AnimatedPressable = Animated.createAnimatedComponent(Pressable); export function ItemCard({ title, subtitle, imageUrl, onPress }: ItemCardProps) { const scale = useSharedValue(1); const animatedStyle = useAnimatedStyle(() => ({ transform: [{ scale: scale.value }], })); return ( <AnimatedPressable style={[styles.card, animatedStyle]} onPress={onPress} onPressIn={() => { scale.value = withSpring(0.97); }} onPressOut={() => { scale.value = withSpring(1); }} > <Image source={{ uri: imageUrl }} style={styles.image} /> <View style={styles.content}> <Text style={styles.title} numberOfLines={1}> {title} </Text> <Text style={styles.subtitle} numberOfLines={2}> {subtitle} </Text> </View> </AnimatedPressable> ); } const styles = StyleSheet.create({ card: { backgroundColor: '#ffffff', borderRadius: 16, overflow: 'hidden', shadowColor: '#000', shadowOffset: { width: 0, height: 2 }, shadowOpacity: 0.1, shadowRadius: 8, elevation: 4, }, image: { width: '100%', height: 160, backgroundColor: '#f3f4f6', }, content: { padding: 16, gap: 4, }, title: { fontSize: 18, fontWeight: '600', color: '#1f2937', }, subtitle: { fontSize: 14, color: '#6b7280', lineHeight: 20, }, }); ``` ## Best Practices 1. **Use TypeScript**: Define navigation and prop types for type safety 2. **Memoize Components**: Use `React.memo` and `useCallback` to prevent unnecessary rerenders 3. **Run Animations on UI Thread**: Use Reanimated worklets for 60fps animations 4. **Avoid Inline Styles**: Use StyleSheet.create for performance 5. **Handle Safe Areas**: Use `SafeAreaView` or `useSafeAreaInsets` 6. **Test on Real Devices**: Simulator/emulator performance differs from real devices 7. **Use FlatList for Lists**: Never use ScrollView with map for long lists 8. **Platform-Specific Code**: Use Platform.select for iOS/Android differences ## Common Issues - **Gesture Conflicts**: Wrap gestures with `GestureDetector` and use `simultaneousHandlers` - **Navigation Type Errors**: Define `ParamList` types for all navigators - **Animation Jank**: Move animations to UI thread with `runOnUI` worklets - **Memory Leaks**: Cancel animations and cleanup in useEffect - **Font Loading**: Use `expo-font` or `react-native-asset` for custom fonts - **Safe Area Issues**: Test on notched devices (iPhone, Android with cutouts) ## Resources - [React Native Documentation](https://reactnative.dev/) - [React Navigation](https://reactnavigation.org/) - [Reanimated Documentation](https://docs.swmansion.com/react-native-reanimated/) - [Gesture Handler](https://docs.swmansion.com/react-native-gesture-handler/) - [Expo Documentation](https://docs.expo.dev/)
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

angular-migration

Migrate from AngularJS to Angular using hybrid mode, incremental

coding
⭐1
# Angular Migration Master AngularJS to Angular migration, including hybrid apps, component conversion, dependency injection changes, and routing migration. ## When to Use This Skill - Migrating AngularJS (1.x) applications to Angular (2+) - Running hybrid AngularJS/Angular applications - Converting directives to components - Modernizing dependency injection - Migrating routing systems - Updating to latest Angular versions - Implementing Angular best practices ## Migration Strategies ### 1. Big Bang (Complete Rewrite) - Rewrite entire app in Angular - Parallel development - Switch over at once - **Best for:** Small apps, green field projects ### 2. Incremental (Hybrid Approach) - Run AngularJS and Angular side-by-side - Migrate feature by feature - ngUpgrade for interop - **Best for:** Large apps, continuous delivery ### 3. Vertical Slice - Migrate one feature completely - New features in Angular, maintain old in AngularJS - Gradually replace - **Best for:** Medium apps, distinct features ## Hybrid App Setup ```typescript // main.ts - Bootstrap hybrid app import { platformBrowserDynamic } from "@angular/platform-browser-dynamic"; import { UpgradeModule } from "@angular/upgrade/static"; import { AppModule } from "./app/app.module"; platformBrowserDynamic() .bootstrapModule(AppModule) .then((platformRef) => { const upgrade = platformRef.injector.get(UpgradeModule); // Bootstrap AngularJS upgrade.bootstrap(document.body, ["myAngularJSApp"], { strictDi: true }); }); ``` ```typescript // app.module.ts import { NgModule } from "@angular/core"; import { BrowserModule } from "@angular/platform-browser"; import { UpgradeModule } from "@angular/upgrade/static"; @NgModule({ imports: [BrowserModule, UpgradeModule], }) export class AppModule { constructor(private upgrade: UpgradeModule) {} ngDoBootstrap() { // Bootstrapped manually in main.ts } } ``` ## Component Migration ### AngularJS Controller β†’ Angular Component ```javascript // Before: AngularJS controller angular .module("myApp") .controller("UserController", function ($scope, UserService) { $scope.user = {}; $scope.loadUser = function (id) { UserService.getUser(id).then(function (user) { $scope.user = user; }); }; $scope.saveUser = function () { UserService.saveUser($scope.user); }; }); ``` ```typescript // After: Angular component import { Component, OnInit } from "@angular/core"; import { UserService } from "./user.service"; @Component({ selector: "app-user", template: ` <div> <h2>{{ user.name }}</h2> <button (click)="saveUser()">Save</button> </div> `, }) export class UserComponent implements OnInit { user: any = {}; constructor(private userService: UserService) {} ngOnInit() { this.loadUser(1); } loadUser(id: number) { this.userService.getUser(id).subscribe((user) => { this.user = user; }); } saveUser() { this.userService.saveUser(this.user); } } ``` ### AngularJS Directive β†’ Angular Component ```javascript // Before: AngularJS directive angular.module("myApp").directive("userCard", function () { return { restrict: "E", scope: { user: "=", onDelete: "&", }, template: ` <div class="card"> <h3>{{ user.name }}</h3> <button ng-click="onDelete()">Delete</button> </div> `, }; }); ``` ```typescript // After: Angular component import { Component, Input, Output, EventEmitter } from "@angular/core"; @Component({ selector: "app-user-card", template: ` <div class="card"> <h3>{{ user.name }}</h3> <button (click)="delete.emit()">Delete</button> </div> `, }) export class UserCardComponent { @Input() user: any; @Output() delete = new EventEmitter<void>(); } // Usage: <app-user-card [user]="user" (delete)="handleDelete()"></app-user-card> ``` ## Service Migration ```javascript // Before: AngularJS service angular.module("myApp").factory("UserService", function ($http) { return { getUser: function (id) { return $http.get("/api/users/" + id); }, saveUser: function (user) { return $http.post("/api/users", user); }, }; }); ``` ```typescript // After: Angular service import { Injectable } from "@angular/core"; import { HttpClient } from "@angular/common/http"; import { Observable } from "rxjs"; @Injectable({ providedIn: "root", }) export class UserService { constructor(private http: HttpClient) {} getUser(id: number): Observable<any> { return this.http.get(`/api/users/${id}`); } saveUser(user: any): Observable<any> { return this.http.post("/api/users", user); } } ``` ## Dependency Injection Changes ### Downgrading Angular β†’ AngularJS ```typescript // Angular service import { Injectable } from "@angular/core"; @Injectable({ providedIn: "root" }) export class NewService { getData() { return "data from Angular"; } } // Make available to AngularJS import { downgradeInjectable } from "@angular/upgrade/static"; angular.module("myApp").factory("newService", downgradeInjectable(NewService)); // Use in AngularJS angular.module("myApp").controller("OldController", function (newService) { console.log(newService.getData()); }); ``` ### Upgrading AngularJS β†’ Angular ```typescript // AngularJS service angular.module('myApp').factory('oldService', function() { return { getData: function() { return 'data from AngularJS'; } }; }); // Make available to Angular import { InjectionToken } from '@angular/core'; export const OLD_SERVICE = new InjectionToken<any>('oldService'); @NgModule({ providers: [ { provide: OLD_SERVICE, useFactory: (i: any) => i.get('oldService'), deps: ['$injector'] } ] }) // Use in Angular @Component({...}) export class NewComponent { constructor(@Inject(OLD_SERVICE) private oldService: any) { console.log(this.oldService.getData()); } } ``` ## Routing Migration ```javascript // Before: AngularJS routing angular.module("myApp").config(function ($routeProvider) { $routeProvider .when("/users", { template: "<user-list></user-list>", }) .when("/users/:id", { template: "<user-detail></user-detail>", }); }); ``` ```typescript // After: Angular routing import { NgModule } from "@angular/core"; import { RouterModule, Routes } from "@angular/router"; const routes: Routes = [ { path: "users", component: UserListComponent }, { path: "users/:id", component: UserDetailComponent }, ]; @NgModule({ imports: [RouterModule.forRoot(routes)], exports: [RouterModule], }) export class AppRoutingModule {} ``` ## Forms Migration ```html <!-- Before: AngularJS --> <form name="userForm" ng-submit="saveUser()"> <input type="text" ng-model="user.name" required /> <input type="email" ng-model="user.email" required /> <button ng-disabled="userForm.$invalid">Save</button> </form> ``` ```typescript // After: Angular (Template-driven) @Component({ template: ` <form #userForm="ngForm" (ngSubmit)="saveUser()"> <input type="text" [(ngModel)]="user.name" name="name" required> <input type="email" [(ngModel)]="user.email" name="email" required> <button [disabled]="userForm.invalid">Save</button> </form> ` }) // Or Reactive Forms (preferred) import { FormBuilder, FormGroup, Validators } from '@angular/forms'; @Component({ template: ` <form [formGroup]="userForm" (ngSubmit)="saveUser()"> <input formControlName="name"> <input formControlName="email"> <button [disabled]="userForm.invalid">Save</button> </form> ` }) export class UserFormComponent { userForm: FormGroup; constructor(private fb: FormBuilder) { this.userForm = this.fb.group({ name: ['', Validators.required], email: ['', [Validators.required, Validators.email]] }); } saveUser() { console.log(this.userForm.value); } } ``` ## Migration Timeline ``` Phase 1: Setup (1-2 weeks) - Install Angular CLI - Set up hybrid app - Configure build tools - Set up testing Phase 2: Infrastructure (2-4 weeks) - Migrate services - Migrate utilities - Set up routing - Migrate shared components Phase 3: Feature Migration (varies) - Migrate feature by feature - Test thoroughly - Deploy incrementally Phase 4: Cleanup (1-2 weeks) - Remove AngularJS code - Remove ngUpgrade - Optimize bundle - Final testing ``` ## Resources - **references/hybrid-mode.md**: Hybrid app patterns - **references/component-migration.md**: Component conversion guide - **references/dependency-injection.md**: DI migration strategies - **references/routing.md**: Routing migration - **assets/hybrid-bootstrap.ts**: Hybrid app template - **assets/migration-timeline.md**: Project planning - **scripts/analyze-angular-app.sh**: App analysis script ## Best Practices 1. **Start with Services**: Migrate services first (easier) 2. **Incremental Approach**: Feature-by-feature migration 3. **Test Continuously**: Test at every step 4. **Use TypeScript**: Migrate to TypeScript early 5. **Follow Style Guide**: Angular style guide from day 1 6. **Optimize Later**: Get it working, then optimize 7. **Document**: Keep migration notes ## Common Pitfalls - Not setting up hybrid app correctly - Migrating UI before logic - Ignoring change detection differences - Not handling scope properly - Mixing patterns (AngularJS + Angular) - Inadequate testing
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered