Skip to main content
EVOKORE// BROWSE
>

./browse/prompts

13 NODES
šŸ¤–system prompt•6 months ago

Multi-Server MCP Aggregation Pattern

Aggregate tools across multiple MCP child servers with prefixing, collision avoidance, and routing rules that stay deterministic.

architecture
⭐1
# Multi-Server MCP Aggregation Pattern Imported from curated first-party documentation sources. ## What this covers Use this pattern when an MCP host must broker tools from multiple child servers without sacrificing clarity or control. ## Use this when - Combining tools from several MCP backends - Avoiding tool-name collisions across providers - Keeping origin and routing visible during execution ## Expected outcomes - Server-prefixed names make tool origins obvious - Collisions are avoided without brittle manual renaming - Operators can extend the tool surface without losing determinism ## Source synthesis - EVOKORE-MCP/docs/AGENT33_IMPROVEMENT_INSTRUCTIONS.md (https://github.com/mattmre/EVOKORE-MCP/blob/main/docs/AGENT33_IMPROVEMENT_INSTRUCTIONS.md) - EVOKORE-MCP/docs/TOOLS_AND_DISCOVERY.md (https://github.com/mattmre/EVOKORE-MCP/blob/main/docs/TOOLS_AND_DISCOVERY.md) ## Dedupe notes Uses the improvement-transfer doc as the primary source, with the discovery doc covering prefixing and compatibility details. ## Source excerpts ### EVOKORE-MCP/docs/AGENT33_IMPROVEMENT_INSTRUCTIONS.md > **Purpose**: Feed this file into Claude Code CLI when working on the Agent33 repo. It contains patterns, architectures, and capabilities proven in EVOKORE-MCP that Agent33 should adopt. --- ## 1. Multi-Server MCP Aggregation Pattern **What Agent33 lacks**: Agent33's MCP server (Phase 43) is a single-endpoint bridge. It doesn't aggregate multiple child MCP servers behind a unified namespace. **What to build**: A proxy layer that spawns and manages multiple child MCP servers from a single config file, presenting them as one unified tool surface. ### Implementation spec: ``` mcp.config.json { "servers": { "github": { "command": "npx", "args": ["-y", "@modelcontextprotocol/server-github"], "env": { "GITHUB_TOKEN": "${GITHUB_TOKEN}" } }, "fs": { "command": "npx", "args": ["-y", "@modelcontextprotocol/server-filesystem", "./"] }, "elevenlabs": { "command": "uvx", "args": ["elevenlabs-mcp"], "env": { "ELEVENLABS_API_KEY": "${ELEVENLABS_API_KEY}" } } } } ``` **Key patterns from EVOKORE**: - **Tool name prefixing**: Every proxied tool gets renamed `{serverId}_{originalName}` to prevent namespace collisions (e.g., `github_create_issue`, `fs_read_file`). First-registration-wins for duplicates. - **Environment interpolation**: `${VAR}` syntax in `env` blocks resolved ... ### EVOKORE-MCP/docs/TOOLS_AND_DISCOVERY.md This page explains how EVOKORE presents tools, how proxy names are built, and how `discover_tools` changes the visible tool surface. ## Two tool populations ### Native EVOKORE tools These tools are defined by EVOKORE itself: - `docs_architect` - `skill_creator` - `resolve_workflow` - `search_skills` - `get_skill_help` - `discover_tools` Properties: - always available - always visible - not subject to proxy prefixing ### Proxied child-server tools These come from child servers in `mcp.config.json`. Current configured sources: - `github` - `fs` - optional `elevenlabs` Properties: - fetched from child servers at startup - renamed with server prefixes - governed by `permissions.yml` - routed through `ProxyManager` ## Prefixing and compatibility EVOKORE rewrites proxied tool names to: ```text ${serverId}_${tool.name} ``` Why this exists: - prevents tool-name collisions across child servers - makes origin obvious during execution and review - keeps exact-name routing deterministic Examples: | Upstream tool | EVOKORE-exposed tool | |---|---| | `read_file` from `fs` | `fs_read_file` | | `create_issue` from `github` | `github_create_issue` | ### Duplicate-prefixed name policy If two child registrations would create the same final prefixed name: - the first registrati ...
šŸ‘0
šŸ‘ļø0
docs
šŸ¤–system prompt•6 months ago

Dynamic Tool Discovery and Session Activation

Expose only native tools at session start, then activate proxied tools on demand through a searchable discovery layer.

architecture
⭐1
# Dynamic Tool Discovery and Session Activation Imported from curated first-party documentation sources. ## What this covers Use this skill when you need smaller tool payloads, exact-name compatibility, and session-scoped tool activation. ## Use this when - Reducing tool-list bloat for focused sessions - Preserving exact-name tool calls while hiding noise - Activating proxied tools only when a task requires them ## Expected outcomes - Discovery changes listing visibility without breaking routing - Session-scoped activation keeps tools relevant to the task - Operators can trade compatibility against payload size intentionally ## Source synthesis - EVOKORE-MCP/docs/TOOLS_AND_DISCOVERY.md (https://github.com/mattmre/EVOKORE-MCP/blob/main/docs/TOOLS_AND_DISCOVERY.md) ## Dedupe notes Uses the canonical EVOKORE-MCP discovery doc instead of duplicating the same concept from walkthrough and migration docs. ## Source excerpts ### EVOKORE-MCP/docs/TOOLS_AND_DISCOVERY.md This page explains how EVOKORE presents tools, how proxy names are built, and how `discover_tools` changes the visible tool surface. ## Two tool populations ### Native EVOKORE tools These tools are defined by EVOKORE itself: - `docs_architect` - `skill_creator` - `resolve_workflow` - `search_skills` - `get_skill_help` - `discover_tools` Properties: - always available - always visible - not subject to proxy prefixing ### Proxied child-server tools These come from child servers in `mcp.config.json`. Current configured sources: - `github` - `fs` - optional `elevenlabs` Properties: - fetched from child servers at startup - renamed with server prefixes - governed by `permissions.yml` - routed through `ProxyManager` ## Prefixing and compatibility EVOKORE rewrites proxied tool names to: ```text ${serverId}_${tool.name} ``` Why this exists: - prevents tool-name collisions across child servers - makes origin obvious during execution and review - keeps exact-name routing deterministic Examples: | Upstream tool | EVOKORE-exposed tool | |---|---| | `read_file` from `fs` | `fs_read_file` | | `create_issue` from `github` | `github_create_issue` | ### Duplicate-prefixed name policy If two child registrations would create the same final prefixed name: - the first registration wins - later duplicates are skipped - EVOKORE logs a warning and duplicate summary ## Discovery modes | Mode | What `tools/list` returns | Best for | |---|---|---| | `legacy` | all native + proxied tools | maximum compatibility | | `dynamic` | native tools + session-activated proxied tools | smaller initial tool payloads | Environment toggle: ```bash EVOKORE_TOOL_DISCOVERY_MODE=legacy EVOKORE_TOOL_DISCOVERY_MODE=dynamic ``` ## Dynamic discovery lifecycle In `dynamic` mode, EVOKORE uses a session-scoped activation set. Lifecycle: 1. session starts with only native tools visible 2. user/model calls `discover_tools` 3. `ToolCatalogIndex` searches the merged native + proxied catalog 4. matching proxied tools are activated for that session 5. EVOKORE emits `sendToolListChanged()` best-effort 6. client re-runs `tools/list` or auto-refreshes ```mermaid flowchart TD A[Session starts in dynamic mode] --> B[tools/list shows native tools] B --> C[discover_tools query] C --> D[ToolCatalogIndex searches merged catalog] D --> E[Matching proxied tools added to session activation set] E --> F[sendToolListChanged best-effort] F --> G[Client refreshes tools/list] G --> H[Activated proxied tools now visible] D --> I[Exact-name proxied call still works eve ...
šŸ‘0
šŸ‘ļø0
docs
šŸ“text•6 months ago

Docker-Backed Jupyter Kernel Operator

Operate containerized Jupyter kernels with enablement, smoke checks, cleanup, and failure handling baked into the runbook.

devops
⭐1
# Docker-Backed Jupyter Kernel Operator Imported from curated first-party documentation sources. ## What this covers Use this runbook when you need isolated notebook execution, clear container lifecycle management, and troubleshooting guidance. ## Use this when - Enabling containerized notebook workflows - Cleaning up kernel containers safely - Troubleshooting notebook execution environments ## Expected outcomes - Kernel startup and cleanup steps are documented end-to-end - Failure modes are easier to diagnose - Notebook execution can be repeated without manual guesswork ## Source synthesis - AGENT33/docs/runbooks/jupyter-kernel-containers.md (https://github.com/mattmre/AGENT33/blob/main/docs/runbooks/jupyter-kernel-containers.md) ## Dedupe notes Uses the AGENT33 Jupyter container runbook as a focused operations import without duplicating broader walkthroughs. ## Source excerpts ### AGENT33/docs/runbooks/jupyter-kernel-containers.md ## Purpose Operate the Docker-backed Jupyter kernel adapter introduced for Phase 38 Stage 3 / Phase 42 follow-on work. ## Enablement Set: - `JUPYTER_KERNEL_ENABLED=true` - `JUPYTER_KERNEL_MODE=docker` Optional settings: - `JUPYTER_KERNEL_DOCKER_IMAGE` - `JUPYTER_KERNEL_ALLOWED_IMAGES` - `JUPYTER_KERNEL_NETWORK_ENABLED` - `JUPYTER_KERNEL_MOUNT_WORKDIR` - `JUPYTER_KERNEL_CONTAINER_WORKDIR` ## Operational Notes - Docker mode publishes kernel ports to the host and mounts a per-session runtime directory containing the Jupyter connection file. - When `JUPYTER_KERNEL_NETWORK_ENABLED=false`, the adapter starts containers with `--network none`. - Working-directory mounting is opt-in and should only point at paths already approved by workflow / execution policy. - The adapter enforces an image allowlist when one is configured. ## Failure Modes - `jupyter_client not installed`: install with `pip install agent33[jupyter]` - `docker executable not found`: install Docker and ensure `docker` is on `PATH` - `Docker image ... is not permitted`: align the requested image with `JUPYTER_KERNEL_ALLOWED_IMAGES` - kernel startup timeout: inspect Docker logs for the session container and verify the image includes `ipykernel` ## Cleanup - One-shot sessions are removed after execution. - Stateful sessions are removed explicitly or via adapter shutdown. - Forced cleanup uses `docker rm -f <container>` and deletes the runtime connection directory. ## Quick Smoke Workflow Register a minimal workflow that exercises the Docker-backed `code-interpreter` tool: ```bash curl -X POST http://localhost:8000/v1/workflows/ \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{ "name": "docker-kernel-smoke", "version": "1.0.0", "description": "Validate Docker-backed Jupyter execution", "triggers": {"manual": true}, "inputs": {}, "outputs": { "result": {"type": "object"} }, "steps": [ { "id": "run-notebook-code", "action": "execute-code", "inputs": { "tool_id": "code-interpreter", "language": "python", "code": "print(6 * 7)" } } ], "execution": {"mode": "sequential"} }' ``` Then execute it: ```bash curl -X POST http://localhost:8000/v1/workflows/docker-kernel-smoke/execute \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{"inputs": {}}' ```
šŸ‘0
šŸ‘ļø0
docs
šŸ¤–system prompt•7 months ago

python-observability

Python observability patterns including structured logging,

coding
⭐1
# Python Observability Instrument Python applications with structured logs, metrics, and traces. When something breaks in production, you need to answer "what, where, and why" without deploying new code. ## When to Use This Skill - Adding structured logging to applications - Implementing metrics collection with Prometheus - Setting up distributed tracing across services - Propagating correlation IDs through request chains - Debugging production issues - Building observability dashboards ## Core Concepts ### 1. Structured Logging Emit logs as JSON with consistent fields for production environments. Machine-readable logs enable powerful queries and alerts. For local development, consider human-readable formats. ### 2. The Four Golden Signals Track latency, traffic, errors, and saturation for every service boundary. ### 3. Correlation IDs Thread a unique ID through all logs and spans for a single request, enabling end-to-end tracing. ### 4. Bounded Cardinality Keep metric label values bounded. Unbounded labels (like user IDs) explode storage costs. ## Quick Start ```python import structlog structlog.configure( processors=[ structlog.processors.TimeStamper(fmt="iso"), structlog.processors.JSONRenderer(), ], ) logger = structlog.get_logger() logger.info("Request processed", user_id="123", duration_ms=45) ``` ## Fundamental Patterns ### Pattern 1: Structured Logging with Structlog Configure structlog for JSON output with consistent fields. ```python import logging import structlog def configure_logging(log_level: str = "INFO") -> None: """Configure structured logging for the application.""" structlog.configure( processors=[ structlog.contextvars.merge_contextvars, structlog.processors.add_log_level, structlog.processors.TimeStamper(fmt="iso"), structlog.processors.StackInfoRenderer(), structlog.processors.format_exc_info, structlog.processors.JSONRenderer(), ], wrapper_class=structlog.make_filtering_bound_logger( getattr(logging, log_level.upper()) ), context_class=dict, logger_factory=structlog.PrintLoggerFactory(), cache_logger_on_first_use=True, ) # Initialize at application startup configure_logging("INFO") logger = structlog.get_logger() ``` ### Pattern 2: Consistent Log Fields Every log entry should include standard fields for filtering and correlation. ```python import structlog from contextvars import ContextVar # Store correlation ID in context correlation_id: ContextVar[str] = ContextVar("correlation_id", default="") logger = structlog.get_logger() def process_request(request: Request) -> Response: """Process request with structured logging.""" logger.info( "Request received", correlation_id=correlation_id.get(), method=request.method, path=request.path, user_id=request.user_id, ) try: result = handle_request(request) logger.info( "Request completed", correlation_id=correlation_id.get(), status_code=200, duration_ms=elapsed, ) return result except Exception as e: logger.error( "Request failed", correlation_id=correlation_id.get(), error_type=type(e).__name__, error_message=str(e), ) raise ``` ### Pattern 3: Semantic Log Levels Use log levels consistently across the application. | Level | Purpose | Examples | |-------|---------|----------| | `DEBUG` | Development diagnostics | Variable values, internal state | | `INFO` | Request lifecycle, operations | Request start/end, job completion | | `WARNING` | Recoverable anomalies | Retry attempts, fallback used | | `ERROR` | Failures needing attention | Exceptions, service unavailable | ```python # DEBUG: Detailed internal information logger.debug("Cache lookup", key=cache_key, hit=cache_hit) # INFO: Normal operational events logger.info("Order created", order_id=order.id, total=order.total) # WARNING: Abnormal but handled situations logger.warning( "Rate limit approaching", current_rate=950, limit=1000, reset_seconds=30, ) # ERROR: Failures requiring investigation logger.error( "Payment processing failed", order_id=order.id, error=str(e), payment_provider="stripe", ) ``` Never log expected behavior at `ERROR`. A user entering a wrong password is `INFO`, not `ERROR`. ### Pattern 4: Correlation ID Propagation Generate a unique ID at ingress and thread it through all operations. ```python from contextvars import ContextVar import uuid import structlog correlation_id: ContextVar[str] = ContextVar("correlation_id", default="") def set_correlation_id(cid: str | None = None) -> str: """Set correlation ID for current context.""" cid = cid or str(uuid.uuid4()) correlation_id.set(cid) structlog.contextvars.bind_contextvars(correlation_id=cid) return cid # FastAPI middleware example from fastapi import Request async def correlation_middleware(request: Request, call_next): """Middleware to set and propagate correlation ID.""" # Use incoming header or generate new cid = request.headers.get("X-Correlation-ID") or str(uuid.uuid4()) set_correlation_id(cid) response = await call_next(request) response.headers["X-Correlation-ID"] = cid return response ``` Propagate to outbound requests: ```python import httpx async def call_downstream_service(endpoint: str, data: dict) -> dict: """Call downstream service with correlation ID.""" async with httpx.AsyncClient() as client: response = await client.post( endpoint, json=data, headers={"X-Correlation-ID": correlation_id.get()}, ) return response.json() ``` ## Advanced Patterns ### Pattern 5: The Four Golden Signals with Prometheus Track these metrics for every service boundary: ```python from prometheus_client import Counter, Histogram, Gauge # Latency: How long requests take REQUEST_LATENCY = Histogram( "http_request_duration_seconds", "Request latency in seconds", ["method", "endpoint", "status"], buckets=[0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10], ) # Traffic: Request rate REQUEST_COUNT = Counter( "http_requests_total", "Total HTTP requests", ["method", "endpoint", "status"], ) # Errors: Error rate ERROR_COUNT = Counter( "http_errors_total", "Total HTTP errors", ["method", "endpoint", "error_type"], ) # Saturation: Resource utilization DB_POOL_USAGE = Gauge( "db_connection_pool_used", "Number of database connections in use", ) ``` Instrument your endpoints: ```python import time from functools import wraps def track_request(func): """Decorator to track request metrics.""" @wraps(func) async def wrapper(request: Request, *args, **kwargs): method = request.method endpoint = request.url.path start = time.perf_counter() try: response = await func(request, *args, **kwargs) status = str(response.status_code) return response except Exception as e: status = "500" ERROR_COUNT.labels( method=method, endpoint=endpoint, error_type=type(e).__name__, ).inc() raise finally: duration = time.perf_counter() - start REQUEST_COUNT.labels(method=method, endpoint=endpoint, status=status).inc() REQUEST_LATENCY.labels(method=method, endpoint=endpoint, status=status).observe(duration) return wrapper ``` ### Pattern 6: Bounded Cardinality Avoid labels with unbounded values to prevent metric explosion. ```python # BAD: User ID has potentially millions of values REQUEST_COUNT.labels(method="GET", user_id=user.id) # Don't do this! # GOOD: Bounded values only REQUEST_COUNT.labels(method="GET", endpoint="/users", status="200") # If you need per-user metrics, use a different approach: # - Log the user_id and query logs # - Use a separate analytics system # - Bucket users by type/tier REQUEST_COUNT.labels( method="GET", endpoint="/users", user_tier="premium", # Bounded set of values ) ``` ### Pattern 7: Timed Operations with Context Manager Create a reusable timing context manager for operations. ```python from contextlib import contextmanager import time import structlog logger = structlog.get_logger() @contextmanager def timed_operation(name: str, **extra_fields): """Context manager for timing and logging operations.""" start = time.perf_counter() logger.debug("Operation started", operation=name, **extra_fields) try: yield except Exception as e: elapsed_ms = (time.perf_counter() - start) * 1000 logger.error( "Operation failed", operation=name, duration_ms=round(elapsed_ms, 2), error=str(e), **extra_fields, ) raise else: elapsed_ms = (time.perf_counter() - start) * 1000 logger.info( "Operation completed", operation=name, duration_ms=round(elapsed_ms, 2), **extra_fields, ) # Usage with timed_operation("fetch_user_orders", user_id=user.id): orders = await order_repository.get_by_user(user.id) ``` ### Pattern 8: OpenTelemetry Tracing Set up distributed tracing with OpenTelemetry. **Note:** OpenTelemetry is actively evolving. Check the [official Python documentation](https://opentelemetry.io/docs/languages/python/) for the latest API patterns and best practices. ```python from opentelemetry import trace from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import BatchSpanProcessor from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter def configure_tracing(service_name: str, otlp_endpoint: str) -> None: """Configure OpenTelemetry tracing.""" provider = TracerProvider() processor = BatchSpanProcessor(OTLPSpanExporter(endpoint=otlp_endpoint)) provider.add_span_processor(processor) trace.set_tracer_provider(provider) tracer = trace.get_tracer(__name__) async def process_order(order_id: str) -> Order: """Process order with tracing.""" with tracer.start_as_current_span("process_order") as span: span.set_attribute("order.id", order_id) with tracer.start_as_current_span("validate_order"): validate_order(order_id) with tracer.start_as_current_span("charge_payment"): charge_payment(order_id) with tracer.start_as_current_span("send_confirmation"): send_confirmation(order_id) return order ``` ## Best Practices Summary 1. **Use structured logging** - JSON logs with consistent fields 2. **Propagate correlation IDs** - Thread through all requests and logs 3. **Track the four golden signals** - Latency, traffic, errors, saturation 4. **Bound label cardinality** - Never use unbounded values as metric labels 5. **Log at appropriate levels** - Don't cry wolf with ERROR 6. **Include context** - User ID, request ID, operation name in logs 7. **Use context managers** - Consistent timing and error handling 8. **Separate concerns** - Observability code shouldn't pollute business logic 9. **Test your observability** - Verify logs and metrics in integration tests 10. **Set up alerts** - Metrics are useless without alerting
šŸ‘0
šŸ‘ļø0
šŸ¤– Auto-discovered
šŸ¤–system prompt•7 months ago

protocol-reverse-engineering

Master network protocol reverse engineering including packet

security
⭐1
# Protocol Reverse Engineering Comprehensive techniques for capturing, analyzing, and documenting network protocols for security research, interoperability, and debugging. ## Traffic Capture ### Wireshark Capture ```bash # Capture on specific interface wireshark -i eth0 -k # Capture with filter wireshark -i eth0 -k -f "port 443" # Capture to file tshark -i eth0 -w capture.pcap # Ring buffer capture (rotate files) tshark -i eth0 -b filesize:100000 -b files:10 -w capture.pcap ``` ### tcpdump Capture ```bash # Basic capture tcpdump -i eth0 -w capture.pcap # With filter tcpdump -i eth0 port 8080 -w capture.pcap # Capture specific bytes tcpdump -i eth0 -s 0 -w capture.pcap # Full packet # Real-time display tcpdump -i eth0 -X port 80 ``` ### Man-in-the-Middle Capture ```bash # mitmproxy for HTTP/HTTPS mitmproxy --mode transparent -p 8080 # SSL/TLS interception mitmproxy --mode transparent --ssl-insecure # Dump to file mitmdump -w traffic.mitm # Burp Suite # Configure browser proxy to 127.0.0.1:8080 ``` ## Protocol Analysis ### Wireshark Analysis ``` # Display filters tcp.port == 8080 http.request.method == "POST" ip.addr == 192.168.1.1 tcp.flags.syn == 1 && tcp.flags.ack == 0 frame contains "password" # Following streams Right-click > Follow > TCP Stream Right-click > Follow > HTTP Stream # Export objects File > Export Objects > HTTP # Decryption Edit > Preferences > Protocols > TLS - (Pre)-Master-Secret log filename - RSA keys list ``` ### tshark Analysis ```bash # Extract specific fields tshark -r capture.pcap -T fields -e ip.src -e ip.dst -e tcp.port # Statistics tshark -r capture.pcap -q -z conv,tcp tshark -r capture.pcap -q -z endpoints,ip # Filter and extract tshark -r capture.pcap -Y "http" -T json > http_traffic.json # Protocol hierarchy tshark -r capture.pcap -q -z io,phs ``` ### Scapy for Custom Analysis ```python from scapy.all import * # Read pcap packets = rdpcap("capture.pcap") # Analyze packets for pkt in packets: if pkt.haslayer(TCP): print(f"Src: {pkt[IP].src}:{pkt[TCP].sport}") print(f"Dst: {pkt[IP].dst}:{pkt[TCP].dport}") if pkt.haslayer(Raw): print(f"Data: {pkt[Raw].load[:50]}") # Filter packets http_packets = [p for p in packets if p.haslayer(TCP) and (p[TCP].sport == 80 or p[TCP].dport == 80)] # Create custom packets pkt = IP(dst="target")/TCP(dport=80)/Raw(load="GET / HTTP/1.1\r\n") send(pkt) ``` ## Protocol Identification ### Common Protocol Signatures ``` HTTP - "HTTP/1." or "GET " or "POST " at start TLS/SSL - 0x16 0x03 (record layer) DNS - UDP port 53, specific header format SMB - 0xFF 0x53 0x4D 0x42 ("SMB" signature) SSH - "SSH-2.0" banner FTP - "220 " response, "USER " command SMTP - "220 " banner, "EHLO" command MySQL - 0x00 length prefix, protocol version PostgreSQL - 0x00 0x00 0x00 startup length Redis - "*" RESP array prefix MongoDB - BSON documents with specific header ``` ### Protocol Header Patterns ``` +--------+--------+--------+--------+ | Magic number / Signature | +--------+--------+--------+--------+ | Version | Flags | +--------+--------+--------+--------+ | Length | Message Type | +--------+--------+--------+--------+ | Sequence Number / Session ID | +--------+--------+--------+--------+ | Payload... | +--------+--------+--------+--------+ ``` ## Binary Protocol Analysis ### Structure Identification ```python # Common patterns in binary protocols # Length-prefixed message struct Message { uint32_t length; # Total message length uint16_t msg_type; # Message type identifier uint8_t flags; # Flags/options uint8_t reserved; # Padding/alignment uint8_t payload[]; # Variable-length payload }; # Type-Length-Value (TLV) struct TLV { uint8_t type; # Field type uint16_t length; # Field length uint8_t value[]; # Field data }; # Fixed header + variable payload struct Packet { uint8_t magic[4]; # "ABCD" signature uint32_t version; uint32_t payload_len; uint32_t checksum; # CRC32 or similar uint8_t payload[]; }; ``` ### Python Protocol Parser ```python import struct from dataclasses import dataclass @dataclass class MessageHeader: magic: bytes version: int msg_type: int length: int @classmethod def from_bytes(cls, data: bytes): magic, version, msg_type, length = struct.unpack( ">4sHHI", data[:12] ) return cls(magic, version, msg_type, length) def parse_messages(data: bytes): offset = 0 messages = [] while offset < len(data): header = MessageHeader.from_bytes(data[offset:]) payload = data[offset+12:offset+12+header.length] messages.append((header, payload)) offset += 12 + header.length return messages # Parse TLV structure def parse_tlv(data: bytes): fields = [] offset = 0 while offset < len(data): field_type = data[offset] length = struct.unpack(">H", data[offset+1:offset+3])[0] value = data[offset+3:offset+3+length] fields.append((field_type, value)) offset += 3 + length return fields ``` ### Hex Dump Analysis ```python def hexdump(data: bytes, width: int = 16): """Format binary data as hex dump.""" lines = [] for i in range(0, len(data), width): chunk = data[i:i+width] hex_part = ' '.join(f'{b:02x}' for b in chunk) ascii_part = ''.join( chr(b) if 32 <= b < 127 else '.' for b in chunk ) lines.append(f'{i:08x} {hex_part:<{width*3}} {ascii_part}') return '\n'.join(lines) # Example output: # 00000000 48 54 54 50 2f 31 2e 31 20 32 30 30 20 4f 4b 0d HTTP/1.1 200 OK. # 00000010 0a 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20 74 .Content-Type: t ``` ## Encryption Analysis ### Identifying Encryption ```python # Entropy analysis - high entropy suggests encryption/compression import math from collections import Counter def entropy(data: bytes) -> float: if not data: return 0.0 counter = Counter(data) probs = [count / len(data) for count in counter.values()] return -sum(p * math.log2(p) for p in probs) # Entropy thresholds: # < 6.0: Likely plaintext or structured data # 6.0-7.5: Possibly compressed # > 7.5: Likely encrypted or random # Common encryption indicators # - High, uniform entropy # - No obvious structure or patterns # - Length often multiple of block size (16 for AES) # - Possible IV at start (16 bytes for AES-CBC) ``` ### TLS Analysis ```bash # Extract TLS metadata tshark -r capture.pcap -Y "ssl.handshake" \ -T fields -e ip.src -e ssl.handshake.ciphersuite # JA3 fingerprinting (client) tshark -r capture.pcap -Y "ssl.handshake.type == 1" \ -T fields -e ssl.handshake.ja3 # JA3S fingerprinting (server) tshark -r capture.pcap -Y "ssl.handshake.type == 2" \ -T fields -e ssl.handshake.ja3s # Certificate extraction tshark -r capture.pcap -Y "ssl.handshake.certificate" \ -T fields -e x509sat.printableString ``` ### Decryption Approaches ```bash # Pre-master secret log (browser) export SSLKEYLOGFILE=/tmp/keys.log # Configure Wireshark # Edit > Preferences > Protocols > TLS # (Pre)-Master-Secret log filename: /tmp/keys.log # Decrypt with private key (if available) # Only works for RSA key exchange # Edit > Preferences > Protocols > TLS > RSA keys list ``` ## Custom Protocol Documentation ### Protocol Specification Template ```markdown # Protocol Name Specification ## Overview Brief description of protocol purpose and design. ## Transport - Layer: TCP/UDP - Port: XXXX - Encryption: TLS 1.2+ ## Message Format ### Header (12 bytes) | Offset | Size | Field | Description | | ------ | ---- | ------- | ----------------------- | | 0 | 4 | Magic | 0x50524F54 ("PROT") | | 4 | 2 | Version | Protocol version (1) | | 6 | 2 | Type | Message type identifier | | 8 | 4 | Length | Payload length in bytes | ### Message Types | Type | Name | Description | | ---- | --------- | ---------------------- | | 0x01 | HELLO | Connection initiation | | 0x02 | HELLO_ACK | Connection accepted | | 0x03 | DATA | Application data | | 0x04 | CLOSE | Connection termination | ### Type 0x01: HELLO | Offset | Size | Field | Description | | ------ | ---- | ---------- | ------------------------ | | 0 | 4 | ClientID | Unique client identifier | | 4 | 2 | Flags | Connection flags | | 6 | var | Extensions | TLV-encoded extensions | ## State Machine ``` [INIT] --HELLO--> [WAIT_ACK] --HELLO_ACK--> [CONNECTED] | DATA/DATA | [CLOSED] <--CLOSE--+ ``` ## Examples ### Connection Establishment ``` Client -> Server: HELLO (ClientID=0x12345678) Server -> Client: HELLO_ACK (Status=OK) Client -> Server: DATA (payload) ``` ``` ### Wireshark Dissector (Lua) ```lua -- custom_protocol.lua local proto = Proto("custom", "Custom Protocol") -- Define fields local f_magic = ProtoField.string("custom.magic", "Magic") local f_version = ProtoField.uint16("custom.version", "Version") local f_type = ProtoField.uint16("custom.type", "Type") local f_length = ProtoField.uint32("custom.length", "Length") local f_payload = ProtoField.bytes("custom.payload", "Payload") proto.fields = { f_magic, f_version, f_type, f_length, f_payload } -- Message type names local msg_types = { [0x01] = "HELLO", [0x02] = "HELLO_ACK", [0x03] = "DATA", [0x04] = "CLOSE" } function proto.dissector(buffer, pinfo, tree) pinfo.cols.protocol = "CUSTOM" local subtree = tree:add(proto, buffer()) -- Parse header subtree:add(f_magic, buffer(0, 4)) subtree:add(f_version, buffer(4, 2)) local msg_type = buffer(6, 2):uint() subtree:add(f_type, buffer(6, 2)):append_text( " (" .. (msg_types[msg_type] or "Unknown") .. ")" ) local length = buffer(8, 4):uint() subtree:add(f_length, buffer(8, 4)) if length > 0 then subtree:add(f_payload, buffer(12, length)) end end -- Register for TCP port local tcp_table = DissectorTable.get("tcp.port") tcp_table:add(8888, proto) ``` ## Active Testing ### Fuzzing with Boofuzz ```python from boofuzz import * def main(): session = Session( target=Target( connection=TCPSocketConnection("target", 8888) ) ) # Define protocol structure s_initialize("HELLO") s_static(b"\x50\x52\x4f\x54") # Magic s_word(1, name="version") # Version s_word(0x01, name="type") # Type (HELLO) s_size("payload", length=4) # Length field s_block_start("payload") s_dword(0x12345678, name="client_id") s_word(0, name="flags") s_block_end() session.connect(s_get("HELLO")) session.fuzz() if __name__ == "__main__": main() ``` ### Replay and Modification ```python from scapy.all import * # Replay captured traffic packets = rdpcap("capture.pcap") for pkt in packets: if pkt.haslayer(TCP) and pkt[TCP].dport == 8888: send(pkt) # Modify and replay for pkt in packets: if pkt.haslayer(Raw): # Modify payload original = pkt[Raw].load modified = original.replace(b"client", b"CLIENT") pkt[Raw].load = modified # Recalculate checksums del pkt[IP].chksum del pkt[TCP].chksum send(pkt) ``` ## Best Practices ### Analysis Workflow 1. **Capture traffic**: Multiple sessions, different scenarios 2. **Identify boundaries**: Message start/end markers 3. **Map structure**: Fixed header, variable payload 4. **Identify fields**: Compare multiple samples 5. **Document format**: Create specification 6. **Validate understanding**: Implement parser/generator 7. **Test edge cases**: Fuzzing, boundary conditions ### Common Patterns to Look For - Magic numbers/signatures at message start - Version fields for compatibility - Length fields (often before variable data) - Type/opcode fields for message identification - Sequence numbers for ordering - Checksums/CRCs for integrity - Timestamps for timing - Session/connection identifiers
šŸ‘0
šŸ‘ļø0
šŸ¤– Auto-discovered
šŸ¤–system prompt•7 months ago

competitive-landscape

This skill should be used when the user asks to "analyze

business
⭐1
# Competitive Landscape Analysis Comprehensive frameworks for analyzing competition, identifying differentiation opportunities, and developing winning market positioning strategies. ## Overview Understand competitive dynamics using proven frameworks (Porter's Five Forces, Blue Ocean Strategy, positioning maps) to identify opportunities and craft defensible competitive advantages. ## Porter's Five Forces Analyze industry attractiveness and competitive intensity. ### Force 1: Threat of New Entrants **Barriers to Entry:** - Capital requirements - Economies of scale - Switching costs - Brand loyalty - Regulatory barriers - Access to distribution - Network effects **High Threat:** Low barriers, easy to enter (e.g., simple SaaS tools) **Low Threat:** High barriers (e.g., regulated industries, hardware) **Analysis Questions:** - How easy is it for new competitors to enter? - What would it cost to launch a competing product? - Are there network effects or switching costs protecting incumbents? ### Force 2: Bargaining Power of Suppliers **Supplier Power Factors:** - Supplier concentration - Availability of substitutes - Importance to supplier - Switching costs - Forward integration threat **High Power:** Few suppliers, critical inputs (e.g., cloud infrastructure providers) **Low Power:** Many alternatives, commoditized (e.g., generic services) **Analysis Questions:** - Who are our critical suppliers? - Could they raise prices or reduce quality? - Can we switch suppliers easily? ### Force 3: Bargaining Power of Buyers **Buyer Power Factors:** - Buyer concentration - Volume purchased - Product differentiation - Price sensitivity - Backward integration threat **High Power:** Few large customers, standardized products (e.g., enterprise deals) **Low Power:** Many small customers, differentiated product (e.g., consumer subscriptions) **Analysis Questions:** - Can customers easily switch to competitors? - Do few customers generate most revenue? - How price-sensitive are buyers? ### Force 4: Threat of Substitutes **Substitute Considerations:** - Alternative solutions - Price-performance tradeoff - Switching costs - Buyer propensity to substitute **High Threat:** Many alternatives, low switching cost (e.g., productivity software) **Low Threat:** Unique solution, high switching cost (e.g., ERP systems) **Analysis Questions:** - What alternative ways can customers solve this problem? - How do substitutes compare on price and performance? - What's the cost to switch to a substitute? ### Force 5: Competitive Rivalry **Rivalry Intensity Factors:** - Number of competitors - Industry growth rate - Product differentiation - Exit barriers - Strategic stakes **High Rivalry:** Many competitors, slow growth, commoditized (e.g., email marketing) **Low Rivalry:** Few competitors, fast growth, differentiated (e.g., emerging AI tools) **Analysis Questions:** - How many direct competitors exist? - Is the market growing or stagnant? - How differentiated are offerings? - Are competitors competing on price or value? ### Forces Analysis Summary Create a scorecard: | Force | Intensity (1-5) | Impact | Key Factors | | -------------- | --------------- | ------ | --------------------------------- | | New Entrants | 3 | Medium | Low barriers but network effects | | Supplier Power | 2 | Low | Many cloud providers | | Buyer Power | 4 | High | Enterprise customers concentrated | | Substitutes | 3 | Medium | Manual processes alternative | | Rivalry | 4 | High | 10+ direct competitors | **Overall Assessment:** Moderate industry attractiveness with high rivalry and buyer power ## Blue Ocean Strategy Identify uncontested market space through value innovation. ### Four Actions Framework **Eliminate:** What factors can be eliminated that the industry takes for granted? **Reduce:** What factors can be reduced well below industry standard? **Raise:** What factors can be raised well above industry standard? **Create:** What factors can be created that the industry never offered? ### Strategy Canvas Map your offering vs. competitors on key factors. **Example: Budget Hotels** ``` High | ā˜… Traditional Hotels | ā˜… Budget Hotels (new) | Low |___________________________________ Price Luxury Convenience Cleanliness Budget Hotel Strategy: - Eliminate: Luxury amenities, room service - Reduce: Lobby size, staff - Raise: Cleanliness, online booking - Create: Self-service kiosks, mobile app ``` ### Value Innovation Find the sweet spot: Lower cost + higher value **Steps:** 1. Map industry competing factors 2. Identify factors to eliminate/reduce (cost savings) 3. Identify factors to raise/create (differentiation) 4. Validate that combination creates new market space ## Competitive Positioning ### Positioning Map Plot competitors on 2-3 key dimensions. **Example Dimensions:** - Price vs. Features - Complexity vs. Ease of Use - Enterprise vs. SMB Focus - Self-Service vs. High-Touch - Generalist vs. Specialist **How to Create:** 1. Choose 2 dimensions most important to customers 2. Plot all competitors 3. Identify gaps (white space) 4. Validate gap represents real customer need **Example:** ``` High Price | | ā˜… Enterprise A ā˜… Enterprise B | | ā— Our Position (gap) | | ā˜… Competitor C ā˜… Competitor D | Low Price |____________________________________________ Simple Complex ``` ### Differentiation Strategy **How to Differentiate:** 1. **Product Differentiation** - Unique features - Superior performance - Better design/UX - Integration ecosystem 2. **Service Differentiation** - Customer support quality - Onboarding experience - Response time - Success programs 3. **Brand Differentiation** - Trust and reputation - Thought leadership - Community - Values alignment 4. **Price Differentiation** - Premium positioning - Value positioning - Transparent pricing - Flexible packaging ### Positioning Statement Framework ``` For [target customer] Who [statement of need or opportunity] Our product is [product category] That [statement of key benefit] Unlike [primary competitive alternative] Our product [statement of primary differentiation] ``` **Example:** ``` For e-commerce companies Who struggle with email marketing automation Our product is an AI-powered email platform That increases conversion rates by 40% Unlike Klaviyo and Mailchimp Our product uses AI to personalize at scale ``` ## Competitive Intelligence ### Information Gathering **Public Sources:** - Company websites and blogs - Press releases and news - Job postings (hint at strategy) - Customer reviews (G2, Capterra) - Social media and forums - Glassdoor (employee insights) - SEC filings (public companies) - Patent filings **Direct Research:** - Customer interviews - Win/loss analysis - Sales team feedback - Product demos and trials - Conference attendance ### Competitor Profile Template For each key competitor, document: **Company Overview:** - Founded, HQ, funding, size - Leadership team - Company stage and trajectory **Product:** - Core features - Target customers - Pricing and packaging - Technology stack - Recent launches **Go-to-Market:** - Sales model (self-serve, sales-led) - Marketing strategy - Distribution channels - Partnerships **Strengths:** - What they do better than anyone - Key competitive advantages - Market position **Weaknesses:** - Gaps in product - Customer complaints - Operational challenges **Strategy:** - Stated direction - Inferred priorities - Likely next moves ## Competitive Pricing Analysis ### Price Positioning **Premium (Top 25%):** - Superior product/service - Strong brand - High-touch sales - Enterprise focus **Mid-Market (Middle 50%):** - Balanced value - Standard features - Mixed sales model - Broad market **Value (Bottom 25%):** - Basic functionality - Self-service - Cost leadership - High volume, low margin ### Pricing Comparison Matrix | Competitor | Entry Price | Mid Tier | Enterprise | Model | | ------------ | ----------- | -------- | ---------- | ------------ | | Competitor A | $29/mo | $99/mo | Custom | Subscription | | Competitor B | $49/mo | $199/mo | $499/mo | Subscription | | Us | $39/mo | $129/mo | Custom | Subscription | **Analysis:** - Are we priced competitively? - What does our pricing signal? - Are there gaps in our packaging? ## Go-to-Market Strategy ### Market Entry Strategies **Direct Competition:** - Head-to-head against established players - Requires differentiation and resources - Example: Better features at lower price **Niche Focus:** - Target underserved segment - Become specialist vs. generalist - Example: "Salesforce for real estate" **Disruptive Innovation:** - Target non-consumers or low end - Improve over time to move upmarket - Example: Freemium model disrupting enterprise **Platform Play:** - Build ecosystem and network effects - Aggregate complementary services - Example: Marketplace or API platform ### Beachhead Market **Characteristics of Good Beachhead:** - Specific, reachable segment - Acute pain you solve well - Limited competition - Willing to pay - Can lead to expansion **Example:** Instead of "project management software", target "project management for construction teams" ## Competitive Advantage ### Sustainable Advantages **Network Effects:** - Value increases with users - Example: Slack, marketplaces **Switching Costs:** - High cost to change - Example: CRM systems with data **Economies of Scale:** - Unit costs decrease with volume - Example: Cloud infrastructure **Brand:** - Trust and reputation - Example: Security software **Proprietary Technology:** - Patents or trade secrets - Example: Algorithms, data **Regulatory:** - Licenses or approvals - Example: Fintech, healthcare ### Testing Your Advantage Ask: - Can competitors copy this in < 2 years? - Does this matter to customers? - Do we execute this better than anyone? - Is this advantage durable? If "no" to any, it's not a sustainable advantage. ## Competitive Monitoring ### What to Track **Product Changes:** - New features - Pricing changes - Packaging adjustments **Market Signals:** - Funding announcements - Key hires (especially leadership) - Customer wins/losses - Partnerships **Performance Metrics:** - Revenue (if public or disclosed) - Customer count - Growth rate - Market share estimates ### Monitoring Cadence **Weekly:** - Product release notes - News mentions **Monthly:** - Win/loss analysis review - Positioning map updates **Quarterly:** - Deep competitive review - Strategy adjustment **Annually:** - Major strategy reassessment - Market trends analysis ## Additional Resources ### Reference Files - **`references/frameworks-deep-dive.md`** - Detailed application of each framework with worksheets - **`references/intel-sources.md`** - Comprehensive list of competitive intelligence sources ### Example Files - **`examples/competitor-analysis.md`** - Complete competitive analysis for a SaaS startup - **`examples/positioning-workshop.md`** - Step-by-step positioning development process ## Quick Start To analyze competitive landscape: 1. **Identify competitors** - Direct, indirect, and future threats 2. **Apply Porter's Five Forces** - Assess industry attractiveness 3. **Create positioning map** - Visualize competitive space 4. **Profile top 3-5 competitors** - Deep dive on key rivals 5. **Identify differentiation** - What makes you unique 6. **Analyze pricing** - Where do you fit? 7. **Assess advantages** - What's defensible? 8. **Develop strategy** - How to win For detailed frameworks and examples, see `references/` and `examples/`.
šŸ‘0
šŸ‘ļø0
šŸ¤– Auto-discovered
šŸ¤–system prompt•7 months ago

fastapi-templates

Create production-ready FastAPI projects with async patterns,

coding
⭐1
# FastAPI Project Templates Production-ready FastAPI project structures with async patterns, dependency injection, middleware, and best practices for building high-performance APIs. ## When to Use This Skill - Starting new FastAPI projects from scratch - Implementing async REST APIs with Python - Building high-performance web services and microservices - Creating async applications with PostgreSQL, MongoDB - Setting up API projects with proper structure and testing ## Core Concepts ### 1. Project Structure **Recommended Layout:** ``` app/ ā”œā”€ā”€ api/ # API routes │ ā”œā”€ā”€ v1/ │ │ ā”œā”€ā”€ endpoints/ │ │ │ ā”œā”€ā”€ users.py │ │ │ ā”œā”€ā”€ auth.py │ │ │ └── items.py │ │ └── router.py │ └── dependencies.py # Shared dependencies ā”œā”€ā”€ core/ # Core configuration │ ā”œā”€ā”€ config.py │ ā”œā”€ā”€ security.py │ └── database.py ā”œā”€ā”€ models/ # Database models │ ā”œā”€ā”€ user.py │ └── item.py ā”œā”€ā”€ schemas/ # Pydantic schemas │ ā”œā”€ā”€ user.py │ └── item.py ā”œā”€ā”€ services/ # Business logic │ ā”œā”€ā”€ user_service.py │ └── auth_service.py ā”œā”€ā”€ repositories/ # Data access │ ā”œā”€ā”€ user_repository.py │ └── item_repository.py └── main.py # Application entry ``` ### 2. Dependency Injection FastAPI's built-in DI system using `Depends`: - Database session management - Authentication/authorization - Shared business logic - Configuration injection ### 3. Async Patterns Proper async/await usage: - Async route handlers - Async database operations - Async background tasks - Async middleware ## Implementation Patterns ### Pattern 1: Complete FastAPI Application ```python # main.py from fastapi import FastAPI, Depends from fastapi.middleware.cors import CORSMiddleware from contextlib import asynccontextmanager @asynccontextmanager async def lifespan(app: FastAPI): """Application lifespan events.""" # Startup await database.connect() yield # Shutdown await database.disconnect() app = FastAPI( title="API Template", version="1.0.0", lifespan=lifespan ) # CORS middleware app.add_middleware( CORSMiddleware, allow_origins=["*"], allow_credentials=True, allow_methods=["*"], allow_headers=["*"], ) # Include routers from app.api.v1.router import api_router app.include_router(api_router, prefix="/api/v1") # core/config.py from pydantic_settings import BaseSettings from functools import lru_cache class Settings(BaseSettings): """Application settings.""" DATABASE_URL: str SECRET_KEY: str ACCESS_TOKEN_EXPIRE_MINUTES: int = 30 API_V1_STR: str = "/api/v1" class Config: env_file = ".env" @lru_cache() def get_settings() -> Settings: return Settings() # core/database.py from sqlalchemy.ext.asyncio import create_async_engine, AsyncSession from sqlalchemy.ext.declarative import declarative_base from sqlalchemy.orm import sessionmaker from app.core.config import get_settings settings = get_settings() engine = create_async_engine( settings.DATABASE_URL, echo=True, future=True ) AsyncSessionLocal = sessionmaker( engine, class_=AsyncSession, expire_on_commit=False ) Base = declarative_base() async def get_db() -> AsyncSession: """Dependency for database session.""" async with AsyncSessionLocal() as session: try: yield session await session.commit() except Exception: await session.rollback() raise finally: await session.close() ``` ### Pattern 2: CRUD Repository Pattern ```python # repositories/base_repository.py from typing import Generic, TypeVar, Type, Optional, List from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy import select from pydantic import BaseModel ModelType = TypeVar("ModelType") CreateSchemaType = TypeVar("CreateSchemaType", bound=BaseModel) UpdateSchemaType = TypeVar("UpdateSchemaType", bound=BaseModel) class BaseRepository(Generic[ModelType, CreateSchemaType, UpdateSchemaType]): """Base repository for CRUD operations.""" def __init__(self, model: Type[ModelType]): self.model = model async def get(self, db: AsyncSession, id: int) -> Optional[ModelType]: """Get by ID.""" result = await db.execute( select(self.model).where(self.model.id == id) ) return result.scalars().first() async def get_multi( self, db: AsyncSession, skip: int = 0, limit: int = 100 ) -> List[ModelType]: """Get multiple records.""" result = await db.execute( select(self.model).offset(skip).limit(limit) ) return result.scalars().all() async def create( self, db: AsyncSession, obj_in: CreateSchemaType ) -> ModelType: """Create new record.""" db_obj = self.model(**obj_in.dict()) db.add(db_obj) await db.flush() await db.refresh(db_obj) return db_obj async def update( self, db: AsyncSession, db_obj: ModelType, obj_in: UpdateSchemaType ) -> ModelType: """Update record.""" update_data = obj_in.dict(exclude_unset=True) for field, value in update_data.items(): setattr(db_obj, field, value) await db.flush() await db.refresh(db_obj) return db_obj async def delete(self, db: AsyncSession, id: int) -> bool: """Delete record.""" obj = await self.get(db, id) if obj: await db.delete(obj) return True return False # repositories/user_repository.py from app.repositories.base_repository import BaseRepository from app.models.user import User from app.schemas.user import UserCreate, UserUpdate class UserRepository(BaseRepository[User, UserCreate, UserUpdate]): """User-specific repository.""" async def get_by_email(self, db: AsyncSession, email: str) -> Optional[User]: """Get user by email.""" result = await db.execute( select(User).where(User.email == email) ) return result.scalars().first() async def is_active(self, db: AsyncSession, user_id: int) -> bool: """Check if user is active.""" user = await self.get(db, user_id) return user.is_active if user else False user_repository = UserRepository(User) ``` ### Pattern 3: Service Layer ```python # services/user_service.py from typing import Optional from sqlalchemy.ext.asyncio import AsyncSession from app.repositories.user_repository import user_repository from app.schemas.user import UserCreate, UserUpdate, User from app.core.security import get_password_hash, verify_password class UserService: """Business logic for users.""" def __init__(self): self.repository = user_repository async def create_user( self, db: AsyncSession, user_in: UserCreate ) -> User: """Create new user with hashed password.""" # Check if email exists existing = await self.repository.get_by_email(db, user_in.email) if existing: raise ValueError("Email already registered") # Hash password user_in_dict = user_in.dict() user_in_dict["hashed_password"] = get_password_hash(user_in_dict.pop("password")) # Create user user = await self.repository.create(db, UserCreate(**user_in_dict)) return user async def authenticate( self, db: AsyncSession, email: str, password: str ) -> Optional[User]: """Authenticate user.""" user = await self.repository.get_by_email(db, email) if not user: return None if not verify_password(password, user.hashed_password): return None return user async def update_user( self, db: AsyncSession, user_id: int, user_in: UserUpdate ) -> Optional[User]: """Update user.""" user = await self.repository.get(db, user_id) if not user: return None if user_in.password: user_in_dict = user_in.dict(exclude_unset=True) user_in_dict["hashed_password"] = get_password_hash( user_in_dict.pop("password") ) user_in = UserUpdate(**user_in_dict) return await self.repository.update(db, user, user_in) user_service = UserService() ``` ### Pattern 4: API Endpoints with Dependencies ```python # api/v1/endpoints/users.py from fastapi import APIRouter, Depends, HTTPException, status from sqlalchemy.ext.asyncio import AsyncSession from typing import List from app.core.database import get_db from app.schemas.user import User, UserCreate, UserUpdate from app.services.user_service import user_service from app.api.dependencies import get_current_user router = APIRouter() @router.post("/", response_model=User, status_code=status.HTTP_201_CREATED) async def create_user( user_in: UserCreate, db: AsyncSession = Depends(get_db) ): """Create new user.""" try: user = await user_service.create_user(db, user_in) return user except ValueError as e: raise HTTPException(status_code=400, detail=str(e)) @router.get("/me", response_model=User) async def read_current_user( current_user: User = Depends(get_current_user) ): """Get current user.""" return current_user @router.get("/{user_id}", response_model=User) async def read_user( user_id: int, db: AsyncSession = Depends(get_db), current_user: User = Depends(get_current_user) ): """Get user by ID.""" user = await user_service.repository.get(db, user_id) if not user: raise HTTPException(status_code=404, detail="User not found") return user @router.patch("/{user_id}", response_model=User) async def update_user( user_id: int, user_in: UserUpdate, db: AsyncSession = Depends(get_db), current_user: User = Depends(get_current_user) ): """Update user.""" if current_user.id != user_id: raise HTTPException(status_code=403, detail="Not authorized") user = await user_service.update_user(db, user_id, user_in) if not user: raise HTTPException(status_code=404, detail="User not found") return user @router.delete("/{user_id}", status_code=status.HTTP_204_NO_CONTENT) async def delete_user( user_id: int, db: AsyncSession = Depends(get_db), current_user: User = Depends(get_current_user) ): """Delete user.""" if current_user.id != user_id: raise HTTPException(status_code=403, detail="Not authorized") deleted = await user_service.repository.delete(db, user_id) if not deleted: raise HTTPException(status_code=404, detail="User not found") ``` ### Pattern 5: Authentication & Authorization ```python # core/security.py from datetime import datetime, timedelta from typing import Optional from jose import JWTError, jwt from passlib.context import CryptContext from app.core.config import get_settings settings = get_settings() pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto") ALGORITHM = "HS256" def create_access_token(data: dict, expires_delta: Optional[timedelta] = None): """Create JWT access token.""" to_encode = data.copy() if expires_delta: expire = datetime.utcnow() + expires_delta else: expire = datetime.utcnow() + timedelta(minutes=15) to_encode.update({"exp": expire}) encoded_jwt = jwt.encode(to_encode, settings.SECRET_KEY, algorithm=ALGORITHM) return encoded_jwt def verify_password(plain_password: str, hashed_password: str) -> bool: """Verify password against hash.""" return pwd_context.verify(plain_password, hashed_password) def get_password_hash(password: str) -> str: """Hash password.""" return pwd_context.hash(password) # api/dependencies.py from fastapi import Depends, HTTPException, status from fastapi.security import OAuth2PasswordBearer from jose import JWTError, jwt from sqlalchemy.ext.asyncio import AsyncSession from app.core.database import get_db from app.core.security import ALGORITHM from app.core.config import get_settings from app.repositories.user_repository import user_repository oauth2_scheme = OAuth2PasswordBearer(tokenUrl=f"{settings.API_V1_STR}/auth/login") async def get_current_user( db: AsyncSession = Depends(get_db), token: str = Depends(oauth2_scheme) ): """Get current authenticated user.""" credentials_exception = HTTPException( status_code=status.HTTP_401_UNAUTHORIZED, detail="Could not validate credentials", headers={"WWW-Authenticate": "Bearer"}, ) try: payload = jwt.decode(token, settings.SECRET_KEY, algorithms=[ALGORITHM]) user_id: int = payload.get("sub") if user_id is None: raise credentials_exception except JWTError: raise credentials_exception user = await user_repository.get(db, user_id) if user is None: raise credentials_exception return user ``` ## Testing ```python # tests/conftest.py import pytest import asyncio from httpx import AsyncClient from sqlalchemy.ext.asyncio import create_async_engine, AsyncSession from sqlalchemy.orm import sessionmaker from app.main import app from app.core.database import get_db, Base TEST_DATABASE_URL = "sqlite+aiosqlite:///:memory:" @pytest.fixture(scope="session") def event_loop(): loop = asyncio.get_event_loop_policy().new_event_loop() yield loop loop.close() @pytest.fixture async def db_session(): engine = create_async_engine(TEST_DATABASE_URL, echo=True) async with engine.begin() as conn: await conn.run_sync(Base.metadata.create_all) AsyncSessionLocal = sessionmaker( engine, class_=AsyncSession, expire_on_commit=False ) async with AsyncSessionLocal() as session: yield session @pytest.fixture async def client(db_session): async def override_get_db(): yield db_session app.dependency_overrides[get_db] = override_get_db async with AsyncClient(app=app, base_url="http://test") as client: yield client # tests/test_users.py import pytest @pytest.mark.asyncio async def test_create_user(client): response = await client.post( "/api/v1/users/", json={ "email": "test@example.com", "password": "testpass123", "name": "Test User" } ) assert response.status_code == 201 data = response.json() assert data["email"] == "test@example.com" assert "id" in data ``` ## Resources - **references/fastapi-architecture.md**: Detailed architecture guide - **references/async-best-practices.md**: Async/await patterns - **references/testing-strategies.md**: Comprehensive testing guide - **assets/project-template/**: Complete FastAPI project - **assets/docker-compose.yml**: Development environment setup ## Best Practices 1. **Async All The Way**: Use async for database, external APIs 2. **Dependency Injection**: Leverage FastAPI's DI system 3. **Repository Pattern**: Separate data access from business logic 4. **Service Layer**: Keep business logic out of routes 5. **Pydantic Schemas**: Strong typing for request/response 6. **Error Handling**: Consistent error responses 7. **Testing**: Test all layers independently ## Common Pitfalls - **Blocking Code in Async**: Using synchronous database drivers - **No Service Layer**: Business logic in route handlers - **Missing Type Hints**: Loses FastAPI's benefits - **Ignoring Sessions**: Not properly managing database sessions - **No Testing**: Skipping integration tests - **Tight Coupling**: Direct database access in routes
šŸ‘0
šŸ‘ļø0
šŸ¤– Auto-discovered
šŸ¤–system prompt•7 months ago

startup-financial-modeling

This skill should be used when the user asks to "create financial

business
⭐1
# Startup Financial Modeling Build comprehensive 3-5 year financial models with revenue projections, cost structures, cash flow analysis, and scenario planning for early-stage startups. ## Overview Financial modeling provides the quantitative foundation for startup strategy, fundraising, and operational planning. Create realistic projections using cohort-based revenue modeling, detailed cost structures, and scenario analysis to support decision-making and investor presentations. ## Core Components ### Revenue Model **Cohort-Based Projections:** Build revenue from customer acquisition and retention by cohort. **Formula:** ``` MRR = Ī£ (Cohort Size Ɨ Retention Rate Ɨ ARPU) ARR = MRR Ɨ 12 ``` **Key Inputs:** - Monthly new customer acquisitions - Customer retention rates by month - Average revenue per user (ARPU) - Pricing and packaging assumptions - Expansion revenue (upsells, cross-sells) ### Cost Structure **Operating Expenses Categories:** 1. **Cost of Goods Sold (COGS)** - Hosting and infrastructure - Payment processing fees - Customer support (variable portion) - Third-party services per customer 2. **Sales & Marketing (S&M)** - Customer acquisition cost (CAC) - Marketing programs and advertising - Sales team compensation - Marketing tools and software 3. **Research & Development (R&D)** - Engineering team compensation - Product management - Design and UX - Development tools and infrastructure 4. **General & Administrative (G&A)** - Executive team - Finance, legal, HR - Office and facilities - Insurance and compliance ### Cash Flow Analysis **Components:** - Beginning cash balance - Cash inflows (revenue, fundraising) - Cash outflows (operating expenses, CapEx) - Ending cash balance - Monthly burn rate - Runway (months of cash remaining) **Formula:** ``` Runway = Current Cash Balance / Monthly Burn Rate Monthly Burn = Monthly Revenue - Monthly Expenses ``` ### Headcount Planning **Role-Based Hiring Plan:** Track headcount by department and role. **Key Metrics:** - Fully-loaded cost per employee - Revenue per employee - Headcount by department (% of total) **Typical Ratios (Early-Stage SaaS):** - Engineering: 40-50% - Sales & Marketing: 25-35% - G&A: 10-15% - Customer Success: 5-10% ## Financial Model Structure ### Three-Scenario Framework **Conservative Scenario (P10):** - Slower customer acquisition - Lower pricing or conversion - Higher churn rates - Extended sales cycles - Used for cash management **Base Scenario (P50):** - Most likely outcomes - Realistic assumptions - Primary planning scenario - Used for board reporting **Optimistic Scenario (P90):** - Faster growth - Better unit economics - Lower churn - Used for upside planning ### Time Horizon **Detailed Projections: 3 Years** - Monthly detail for Year 1 - Monthly detail for Year 2 - Quarterly detail for Year 3 **High-Level Projections: Years 4-5** - Annual projections - Key metrics only - Support long-term planning ## Step-by-Step Process ### Step 1: Define Business Model Clarify revenue model and pricing. **SaaS Model:** - Subscription pricing tiers - Annual vs. monthly contracts - Free trial or freemium approach - Expansion revenue strategy **Marketplace Model:** - GMV projections - Take rate (% of transactions) - Buyer and seller economics - Transaction frequency **Transactional Model:** - Transaction volume - Revenue per transaction - Frequency and seasonality ### Step 2: Build Revenue Projections Use cohort-based methodology for accuracy. **Monthly Customer Acquisition:** Define new customers acquired each month. **Retention Curve:** Model customer retention over time. **Typical SaaS Retention:** - Month 1: 100% - Month 3: 90% - Month 6: 85% - Month 12: 75% - Month 24: 70% **Revenue Calculation:** For each cohort, calculate retained customers Ɨ ARPU for each month. ### Step 3: Model Cost Structure Break down costs by category and behavior. **Fixed vs. Variable:** - Fixed: Salaries, software, rent - Variable: Hosting, payment processing, support **Scaling Assumptions:** - COGS as % of revenue - S&M as % of revenue (CAC payback) - R&D growth rate - G&A as % of total expenses ### Step 4: Create Hiring Plan Model headcount growth by role and department. **Inputs:** - Starting headcount - Hiring velocity by role - Fully-loaded compensation by role - Benefits and taxes (typically 1.3-1.4x salary) **Example:** ``` Engineer: $150K salary Ɨ 1.35 = $202K fully-loaded Sales Rep: $100K OTE Ɨ 1.30 = $130K fully-loaded ``` ### Step 5: Project Cash Flow Calculate monthly cash position and runway. **Monthly Cash Flow:** ``` Beginning Cash + Revenue Collected (consider payment terms) - Operating Expenses Paid - CapEx = Ending Cash ``` **Runway Calculation:** ``` If Ending Cash < 0: Funding Need = Negative Cash Balance Runway = 0 Else: Runway = Ending Cash / Average Monthly Burn ``` ### Step 6: Calculate Key Metrics Track metrics that matter for stage. **Revenue Metrics:** - MRR / ARR - Growth rate (MoM, YoY) - Revenue by segment or cohort **Unit Economics:** - CAC (Customer Acquisition Cost) - LTV (Lifetime Value) - CAC Payback Period - LTV / CAC Ratio **Efficiency Metrics:** - Burn multiple (Net Burn / Net New ARR) - Magic number (Net New ARR / S&M Spend) - Rule of 40 (Growth % + Profit Margin %) **Cash Metrics:** - Monthly burn rate - Runway (months) - Cash efficiency ### Step 7: Scenario Analysis Create three scenarios with different assumptions. **Variable Assumptions:** - Customer acquisition rate (±30%) - Churn rate (±20%) - Average contract value (±15%) - CAC (±25%) **Fixed Assumptions:** - Pricing structure - Core operating expenses - Hiring plan (adjust timing, not roles) ## Business Model Templates ### SaaS Financial Model **Revenue Drivers:** - New MRR (customers Ɨ ARPU) - Expansion MRR (upsells) - Contraction MRR (downgrades) - Churned MRR (lost customers) **Key Ratios:** - Gross margin: 75-85% - S&M as % revenue: 40-60% (early stage) - CAC payback: < 12 months - Net retention: 100-120% **Example Projection:** ``` Year 1: $500K ARR, 50 customers, $100K MRR by Dec Year 2: $2.5M ARR, 200 customers, $208K MRR by Dec Year 3: $8M ARR, 600 customers, $667K MRR by Dec ``` ### Marketplace Financial Model **Revenue Drivers:** - GMV (Gross Merchandise Value) - Take rate (% of GMV) - Net revenue = GMV Ɨ Take rate **Key Ratios:** - Take rate: 10-30% depending on category - CAC for buyers vs. sellers - Contribution margin: 60-70% **Example Projection:** ``` Year 1: $5M GMV, 15% take rate = $750K revenue Year 2: $20M GMV, 15% take rate = $3M revenue Year 3: $60M GMV, 15% take rate = $9M revenue ``` ### E-Commerce Financial Model **Revenue Drivers:** - Traffic (visitors) - Conversion rate - Average order value (AOV) - Purchase frequency **Key Ratios:** - Gross margin: 40-60% - Contribution margin: 20-35% - CAC payback: 3-6 months ### Services / Agency Financial Model **Revenue Drivers:** - Billable hours or projects - Hourly rate or project fee - Utilization rate - Team capacity **Key Ratios:** - Gross margin: 50-70% - Utilization: 70-85% - Revenue per employee ## Fundraising Integration ### Funding Scenario Modeling **Pre-Money Valuation:** Based on metrics and comparables. **Dilution:** ``` Post-Money = Pre-Money + Investment Dilution % = Investment / Post-Money ``` **Use of Funds:** Allocate funding to extend runway and achieve milestones. **Example:** ``` Raise: $5M at $20M pre-money Post-Money: $25M Dilution: 20% Use of Funds: - Product Development: $2M (40%) - Sales & Marketing: $2M (40%) - G&A and Operations: $0.5M (10%) - Working Capital: $0.5M (10%) ``` ### Milestone-Based Planning **Identify Key Milestones:** - Product launch - First $1M ARR - Break-even on CAC - Series A fundraise **Funding Amount:** Ensure runway to achieve next milestone + 6 months buffer. ## Common Pitfalls **Pitfall 1: Overly Optimistic Revenue** - New startups rarely hit aggressive projections - Use conservative customer acquisition assumptions - Model realistic churn rates **Pitfall 2: Underestimating Costs** - Add 20% buffer to expense estimates - Include fully-loaded compensation - Account for software and tools **Pitfall 3: Ignoring Cash Flow Timing** - Revenue ≠ cash (payment terms) - Expenses paid before revenue collected - Model cash conversion carefully **Pitfall 4: Static Headcount** - Hiring takes time (3-6 months to fill roles) - Ramp time for productivity (3-6 months) - Account for attrition (10-15% annually) **Pitfall 5: Not Scenario Planning** - Single scenario is never accurate - Always model conservative case - Plan for what you'll do if base case fails ## Model Validation **Sanity Checks:** - [ ] Revenue growth rate is achievable (3x in Year 2, 2x in Year 3) - [ ] Unit economics are realistic (LTV/CAC > 3, payback < 18 months) - [ ] Burn multiple is reasonable (< 2.0 in Year 2-3) - [ ] Headcount scales with revenue (revenue per employee growing) - [ ] Gross margin is appropriate for business model - [ ] S&M spending aligns with CAC and growth targets **Benchmark Against Peers:** Compare key metrics to similar companies at similar stage. **Investor Feedback:** Share model with advisors or investors for feedback on assumptions. ## Additional Resources ### Reference Files For detailed model structures and advanced techniques: - **`references/model-templates.md`** - Complete financial model templates by business model - **`references/unit-economics.md`** - Deep dive on CAC, LTV, payback, and efficiency metrics - **`references/fundraising-scenarios.md`** - Modeling funding rounds and dilution ### Example Files Working financial models with formulas: - **`examples/saas-financial-model.md`** - Complete 3-year SaaS model with cohort analysis - **`examples/marketplace-model.md`** - Marketplace GMV and take rate projections - **`examples/scenario-analysis.md`** - Three-scenario framework with sensitivities ## Quick Start To create a startup financial model: 1. **Define business model** - Revenue drivers and pricing 2. **Project revenue** - Cohort-based with retention 3. **Model costs** - COGS, S&M, R&D, G&A by month 4. **Plan headcount** - Hiring by role and department 5. **Calculate cash flow** - Revenue - expenses = burn/runway 6. **Compute metrics** - CAC, LTV, burn multiple, runway 7. **Create scenarios** - Conservative, base, optimistic 8. **Validate assumptions** - Sanity check and benchmark 9. **Integrate fundraising** - Model funding rounds and milestones For complete templates and formulas, reference the `references/` and `examples/` files.
šŸ‘0
šŸ‘ļø0
šŸ¤– Auto-discovered
šŸ¤–system prompt•7 months ago

startup-metrics-framework

This skill should be used when the user asks about "key startup

business
⭐1
# Startup Metrics Framework Comprehensive guide to tracking, calculating, and optimizing key performance metrics for different startup business models from seed through Series A. ## Overview Track the right metrics at the right stage. Focus on unit economics, growth efficiency, and cash management metrics that matter for fundraising and operational excellence. ## Universal Startup Metrics ### Revenue Metrics **MRR (Monthly Recurring Revenue)** ``` MRR = Ī£ (Active Subscriptions Ɨ Monthly Price) ``` **ARR (Annual Recurring Revenue)** ``` ARR = MRR Ɨ 12 ``` **Growth Rate** ``` MoM Growth = (This Month MRR - Last Month MRR) / Last Month MRR YoY Growth = (This Year ARR - Last Year ARR) / Last Year ARR ``` **Target Benchmarks:** - Seed stage: 15-20% MoM growth - Series A: 10-15% MoM growth, 3-5x YoY - Series B+: 100%+ YoY (Rule of 40) ### Unit Economics **CAC (Customer Acquisition Cost)** ``` CAC = Total S&M Spend / New Customers Acquired ``` Include: Sales salaries, marketing spend, tools, overhead **LTV (Lifetime Value)** ``` LTV = ARPU Ɨ Gross Margin% Ɨ (1 / Churn Rate) ``` Simplified: ``` LTV = ARPU Ɨ Average Customer Lifetime Ɨ Gross Margin% ``` **LTV:CAC Ratio** ``` LTV:CAC = LTV / CAC ``` **Benchmarks:** - LTV:CAC > 3.0 = Healthy - LTV:CAC 1.0-3.0 = Needs improvement - LTV:CAC < 1.0 = Unsustainable **CAC Payback Period** ``` CAC Payback = CAC / (ARPU Ɨ Gross Margin%) ``` **Benchmarks:** - < 12 months = Excellent - 12-18 months = Good - > 24 months = Concerning ### Cash Efficiency Metrics **Burn Rate** ``` Monthly Burn = Monthly Revenue - Monthly Expenses ``` Negative burn = losing money (typical early-stage) **Runway** ``` Runway (months) = Cash Balance / Monthly Burn Rate ``` **Target:** Always maintain 12-18 months runway **Burn Multiple** ``` Burn Multiple = Net Burn / Net New ARR ``` **Benchmarks:** - < 1.0 = Exceptional efficiency - 1.0-1.5 = Good - 1.5-2.0 = Acceptable - > 2.0 = Inefficient Lower is better (spending less to generate ARR) ## SaaS Metrics ### Revenue Composition **New MRR** New customers Ɨ ARPU **Expansion MRR** Upsells and cross-sells from existing customers **Contraction MRR** Downgrades from existing customers **Churned MRR** Lost customers **Net New MRR Formula:** ``` Net New MRR = New MRR + Expansion MRR - Contraction MRR - Churned MRR ``` ### Retention Metrics **Logo Retention** ``` Logo Retention = (Customers End - New Customers) / Customers Start ``` **Dollar Retention (NDR - Net Dollar Retention)** ``` NDR = (ARR Start + Expansion - Contraction - Churn) / ARR Start ``` **Benchmarks:** - NDR > 120% = Best-in-class - NDR 100-120% = Good - NDR < 100% = Needs work **Gross Retention** ``` Gross Retention = (ARR Start - Churn - Contraction) / ARR Start ``` **Benchmarks:** - > 90% = Excellent - 85-90% = Good - < 85% = Concerning ### SaaS-Specific Metrics **Magic Number** ``` Magic Number = Net New ARR (quarter) / S&M Spend (prior quarter) ``` **Benchmarks:** - > 0.75 = Efficient, ready to scale - 0.5-0.75 = Moderate efficiency - < 0.5 = Inefficient, don't scale yet **Rule of 40** ``` Rule of 40 = Revenue Growth Rate% + Profit Margin% ``` **Benchmarks:** - > 40% = Excellent - 20-40% = Acceptable - < 20% = Needs improvement **Example:** 50% growth + (10%) margin = 40% āœ“ **Quick Ratio** ``` Quick Ratio = (New MRR + Expansion MRR) / (Churned MRR + Contraction MRR) ``` **Benchmarks:** - > 4.0 = Healthy growth - 2.0-4.0 = Moderate - < 2.0 = Churn problem ## Marketplace Metrics ### GMV (Gross Merchandise Value) **Total Transaction Volume:** ``` GMV = Ī£ (Transaction Value) ``` **Growth Rate:** ``` GMV Growth Rate = (Current Period GMV - Prior Period GMV) / Prior Period GMV ``` **Target:** 20%+ MoM early-stage ### Take Rate ``` Take Rate = Net Revenue / GMV ``` **Typical Ranges:** - Payment processors: 2-3% - E-commerce marketplaces: 10-20% - Service marketplaces: 15-25% - High-value B2B: 5-15% ### Marketplace Liquidity **Time to Transaction** How long from listing to sale/match? **Fill Rate** % of requests that result in transaction **Repeat Rate** % of users who transact multiple times **Benchmarks:** - Fill rate > 80% = Strong liquidity - Repeat rate > 60% = Strong retention ### Marketplace Balance **Supply/Demand Ratio:** Track relative growth of supply and demand sides. **Warning Signs:** - Too much supply: Low fill rates, frustrated suppliers - Too much demand: Long wait times, frustrated customers **Goal:** Balanced growth (1:1 ratio ideal, but varies by model) ## Consumer/Mobile Metrics ### Engagement Metrics **DAU (Daily Active Users)** Unique users active each day **MAU (Monthly Active Users)** Unique users active each month **DAU/MAU Ratio** ``` DAU/MAU = DAU / MAU ``` **Benchmarks:** - > 50% = Exceptional (daily habit) - 20-50% = Good - < 20% = Weak engagement **Session Frequency** Average sessions per user per day/week **Session Duration** Average time spent per session ### Retention Curves **Day 1 Retention:** % users who return next day **Day 7 Retention:** % users active 7 days after signup **Day 30 Retention:** % users active 30 days after signup **Benchmarks (Day 30):** - > 40% = Excellent - 25-40% = Good - < 25% = Weak **Retention Curve Shape:** - Flattening curve = good (users becoming habitual) - Steep decline = poor product-market fit ### Viral Coefficient (K-Factor) ``` K-Factor = Invites per User Ɨ Invite Conversion Rate ``` **Example:** 10 invites/user Ɨ 20% conversion = 2.0 K-factor **Benchmarks:** - K > 1.0 = Viral growth - K = 0.5-1.0 = Strong referrals - K < 0.5 = Weak virality ## B2B Metrics ### Sales Efficiency **Win Rate** ``` Win Rate = Deals Won / Total Opportunities ``` **Target:** 20-30% for new sales team, 30-40% mature **Sales Cycle Length** Average days from opportunity to close **Shorter is better:** - SMB: 30-60 days - Mid-market: 60-120 days - Enterprise: 120-270 days **Average Contract Value (ACV)** ``` ACV = Total Contract Value / Contract Length (years) ``` ### Pipeline Metrics **Pipeline Coverage** ``` Pipeline Coverage = Total Pipeline Value / Quota ``` **Target:** 3-5x coverage (3-5x pipeline needed to hit quota) **Conversion Rates by Stage:** - Lead → Opportunity: 10-20% - Opportunity → Demo: 50-70% - Demo → Proposal: 30-50% - Proposal → Close: 20-40% ## Metrics by Stage ### Pre-Seed (Product-Market Fit) **Focus Metrics:** 1. Active users growth 2. User retention (Day 7, Day 30) 3. Core engagement (sessions, features used) 4. Qualitative feedback (NPS, interviews) **Don't worry about:** - Revenue (may be zero) - CAC (not optimizing yet) - Unit economics ### Seed ($500K-$2M ARR) **Focus Metrics:** 1. MRR growth rate (15-20% MoM) 2. CAC and LTV (establish baseline) 3. Gross retention (> 85%) 4. Core product engagement **Start tracking:** - Sales efficiency - Burn rate and runway ### Series A ($2M-$10M ARR) **Focus Metrics:** 1. ARR growth (3-5x YoY) 2. Unit economics (LTV:CAC > 3, payback < 18 months) 3. Net dollar retention (> 100%) 4. Burn multiple (< 2.0) 5. Magic number (> 0.5) **Mature tracking:** - Rule of 40 - Sales efficiency - Pipeline coverage ## Metric Tracking Best Practices ### Data Infrastructure **Requirements:** - Single source of truth (analytics platform) - Real-time or daily updates - Automated calculations - Historical tracking **Tools:** - Mixpanel, Amplitude (product analytics) - ChartMogul, Baremetrics (SaaS metrics) - Looker, Tableau (BI dashboards) ### Reporting Cadence **Daily:** - MRR, active users - Sign-ups, conversions **Weekly:** - Growth rates - Retention cohorts - Sales pipeline **Monthly:** - Full metric suite - Board reporting - Investor updates **Quarterly:** - Trend analysis - Benchmarking - Strategy review ### Common Mistakes **Mistake 1: Vanity Metrics** Don't focus on: - Total users (without retention) - Page views (without engagement) - Downloads (without activation) Focus on actionable metrics tied to value. **Mistake 2: Too Many Metrics** Track 5-7 core metrics intensely, not 50 loosely. **Mistake 3: Ignoring Unit Economics** CAC and LTV are critical even at seed stage. **Mistake 4: Not Segmenting** Break down metrics by customer segment, channel, cohort. **Mistake 5: Gaming Metrics** Optimize for real business outcomes, not dashboard numbers. ## Investor Metrics ### What VCs Want to See **Seed Round:** - MRR growth rate - User retention - Early unit economics - Product engagement **Series A:** - ARR and growth rate - CAC payback < 18 months - LTV:CAC > 3.0 - Net dollar retention > 100% - Burn multiple < 2.0 **Series B+:** - Rule of 40 > 40% - Efficient growth (magic number) - Path to profitability - Market leadership metrics ### Metric Presentation **Dashboard Format:** ``` Current MRR: $250K (↑ 18% MoM) ARR: $3.0M (↑ 280% YoY) CAC: $1,200 | LTV: $4,800 | LTV:CAC = 4.0x NDR: 112% | Logo Retention: 92% Burn: $180K/mo | Runway: 18 months ``` **Include:** - Current value - Growth rate or trend - Context (target, benchmark) ## Additional Resources ### Reference Files - **`references/metric-definitions.md`** - Complete definitions and formulas for 50+ metrics - **`references/benchmarks-by-stage.md`** - Target ranges for each metric by company stage - **`references/calculation-examples.md`** - Step-by-step calculation examples ### Example Files - **`examples/saas-metrics-dashboard.md`** - Complete metrics suite for B2B SaaS company - **`examples/marketplace-metrics.md`** - Marketplace-specific metrics with examples - **`examples/investor-metrics-deck.md`** - How to present metrics for fundraising ## Quick Start To implement startup metrics framework: 1. **Identify business model** - SaaS, marketplace, consumer, B2B 2. **Choose 5-7 core metrics** - Based on stage and model 3. **Establish tracking** - Set up analytics and dashboards 4. **Calculate unit economics** - CAC, LTV, payback 5. **Set targets** - Use benchmarks for goals 6. **Review regularly** - Weekly for core metrics 7. **Share with team** - Align on goals and progress 8. **Update investors** - Monthly/quarterly reporting For detailed definitions, benchmarks, and examples, see `references/` and `examples/`.
šŸ‘0
šŸ‘ļø0
šŸ¤– Auto-discovered
šŸ¤–system prompt•7 months ago

team-composition-analysis

This skill should be used when the user asks to "plan team

business
⭐1
# Team Composition Analysis Design optimal team structures, hiring plans, compensation strategies, and equity allocation for early-stage startups from pre-seed through Series A. ## Overview Build the right team at the right time with appropriate compensation and equity. Plan role-by-role hiring aligned with revenue milestones, budget constraints, and market benchmarks. ## Team Structure by Stage ### Pre-Seed (0-$500K ARR) **Team Size: 2-5 people** **Core Roles:** - Founders (2-3): Product, engineering, business - First engineer (if needed) - Contract roles: Design, marketing **Focus:** Build and validate product-market fit ### Seed ($500K-$2M ARR) **Team Size: 5-15 people** **Key Hires:** - Engineering lead + 2-3 engineers - First sales/business development - Product manager - Marketing/growth lead **Focus:** Scale product and prove repeatable sales ### Series A ($2M-$10M ARR) **Team Size: 15-50 people** **Department Build-Out:** - Engineering (40%): 6-20 people - Sales & Marketing (30%): 5-15 people - Customer Success (10%): 2-5 people - G&A (10%): 2-5 people - Product (10%): 2-5 people **Focus:** Scale revenue and build repeatable processes ## Role-by-Role Planning ### Engineering Team **Pre-Seed:** - Founders write code - 0-1 contract developers **Seed:** - Engineering Lead (first $150K-$180K) - 2-3 Full-Stack Engineers ($120K-$150K) - 1 Frontend or Backend Specialist ($130K-$160K) **Series A:** - VP Engineering ($180K-$250K + equity) - 2-3 Senior Engineers ($150K-$180K) - 3-5 Mid-Level Engineers ($120K-$150K) - 1-2 Junior Engineers ($90K-$120K) - 1 DevOps/Infrastructure ($140K-$170K) ### Sales & Marketing **Pre-Seed:** - Founders do sales - Contract marketing help **Seed:** - First Sales Hire / Head of Sales ($120K-$150K + commission) - Marketing/Growth Lead ($100K-$140K) - SDR or BDR (if B2B) ($50K-$70K + commission) **Series A:** - VP Sales ($150K-$200K + commission + equity) - 3-5 Account Executives ($80K-$120K + commission) - 2-3 SDRs/BDRs ($50K-$70K + commission) - Marketing Manager ($90K-$130K) - Content/Demand Gen ($70K-$100K) ### Product Team **Pre-Seed:** - Founder as product lead **Seed:** - First Product Manager ($120K-$150K) - Contract designer **Series A:** - Head of Product ($150K-$180K) - 1-2 Product Managers ($120K-$150K) - Product Designer ($100K-$140K) - UX Researcher (optional) ($90K-$130K) ### Customer Success **Pre-Seed:** - Founders handle support **Seed:** - First CS hire (optional) ($60K-$90K) **Series A:** - CS Manager ($100K-$130K) - 2-4 CS Representatives ($60K-$90K) - Support Engineer (technical) ($80K-$120K) ### G&A (General & Administrative) **Pre-Seed:** - Contractors (accounting, legal) **Seed:** - Operations/Office Manager ($70K-$100K) - Contract CFO **Series A:** - CFO or Finance Lead ($150K-$200K) - Recruiter ($80K-$120K) - Office Manager / EA ($60K-$90K) ## Compensation Strategy ### Base Salary Benchmarks (US, 2024) **Engineering:** - Junior: $90K-$120K - Mid-Level: $120K-$150K - Senior: $150K-$180K - Staff/Principal: $180K-$220K - Engineering Manager: $160K-$200K - VP Engineering: $180K-$250K **Sales:** - SDR/BDR: $50K-$70K base + $50K-$70K commission - Account Executive: $80K-$120K base + $80K-$120K commission - Sales Manager: $120K-$160K base + $80K-$120K commission - VP Sales: $150K-$200K base + $150K-$200K commission **Product:** - Product Manager: $120K-$150K - Senior PM: $150K-$180K - Head of Product: $150K-$180K - VP Product: $180K-$220K **Marketing:** - Marketing Manager: $90K-$130K - Content/Demand Gen: $70K-$100K - Head of Marketing: $130K-$170K - VP Marketing: $150K-$200K **Customer Success:** - CS Representative: $60K-$90K - CS Manager: $100K-$130K - VP Customer Success: $140K-$180K ### Total Compensation Formula ``` Total Comp = Base Salary Ɨ 1.30 (benefits & taxes) + Equity Value ``` **Fully-Loaded Cost:** - Base salary - Payroll taxes (7.65% FICA) - Benefits (health insurance, 401k): $10K-$15K per employee - Other (workspace, equipment, software): $5K-$10K per employee **Rule of Thumb:** Multiply base salary by 1.3-1.4 for fully-loaded cost ### Geographic Adjustments **San Francisco / New York:** +20-30% above benchmarks **Seattle / Boston / Los Angeles:** +10-20% **Austin / Denver / Chicago:** +0-10% **Remote / Other US Cities:** -10-20% **International:** Varies widely by country ## Equity Allocation ### Equity by Role and Stage **Founders:** - First founder: 40-60% - Second founder: 20-40% - Third founder: 10-20% - Vesting: 4 years with 1-year cliff **Early Employees (Pre-Seed):** - First engineer: 0.5-2.0% - First 5 employees: 0.25-1.0% each **Seed Stage Hires:** - VP/Head level: 0.5-1.5% - Senior IC: 0.1-0.5% - Mid-level: 0.05-0.25% - Junior: 0.01-0.1% **Series A Hires:** - C-level (CTO, CFO): 1.0-3.0% - VP level: 0.3-1.0% - Director level: 0.1-0.5% - Senior IC: 0.05-0.2% - Mid-level: 0.01-0.1% - Junior: 0.005-0.05% ### Equity Pool Sizing **Option Pool by Round:** - Pre-Seed: 10-15% reserved - Seed: 10-15% top-up - Series A: 10-15% top-up - Series B+: 5-10% per round **Pre-Funding Dilution:** Investors often require option pool creation before investment, diluting founders. **Example:** ``` Pre-money: $10M Investors want 15% option pool post-money Calculation: Post-money: $15M ($10M + $5M investment) Option pool: $2.25M (15% Ɨ $15M) Founders diluted by pool creation before new money ``` ## Organizational Design ### Reporting Structure **Pre-Seed:** ``` Founders (flat structure) ā”œā”€ā”€ Contractors └── First hires (report to founders) ``` **Seed:** ``` CEO ā”œā”€ā”€ Engineering Lead (2-4 engineers) ā”œā”€ā”€ Sales/Growth Lead (1-2 reps) ā”œā”€ā”€ Product Manager └── Operations ``` **Series A:** ``` CEO ā”œā”€ā”€ CTO / VP Engineering (6-20 people) │ ā”œā”€ā”€ Engineering Manager(s) │ └── Individual Contributors ā”œā”€ā”€ VP Sales (5-15 people) │ ā”œā”€ā”€ Sales Manager │ ā”œā”€ā”€ Account Executives │ └── SDRs ā”œā”€ā”€ Head of Product (2-5 people) │ ā”œā”€ā”€ Product Managers │ └── Designers ā”œā”€ā”€ Head of Customer Success (2-5 people) └── CFO / Finance Lead (2-5 people) ā”œā”€ā”€ Recruiter └── Operations ``` ### Span of Control **Manager Ratios:** - First-line managers: 4-8 direct reports - Directors: 3-5 direct reports (managers) - VPs: 3-5 direct reports (directors) - CEO: 5-8 direct reports (executive team) ## Full-Time vs. Contract ### Use Full-Time for: - Core product development - Sales (revenue-generating roles) - Mission-critical operations - Institutional knowledge roles ### Use Contractors for: - Specialized short-term needs (legal, accounting) - Variable workload (design, marketing campaigns) - Skills outside core competency - Testing role before FTE hire - Geographic expansion before permanent presence ### Cost Comparison **Full-Time:** - Lower hourly cost - Benefits and overhead - Long-term commitment - Cultural fit matters **Contract:** - Higher hourly rate ($75-$200/hour vs. $40-$100/hour FTE equivalent) - No benefits or overhead - Flexible engagement - Easier to scale up/down ## Hiring Velocity ### Realistic Timeline **Role Opening to Hire:** - Junior: 6-8 weeks - Mid-Level: 8-12 weeks - Senior: 12-16 weeks - Executive: 16-24 weeks **Time to Productivity:** - Junior: 4-6 months - Mid-Level: 2-4 months - Senior: 1-3 months - Executive: 3-6 months ### Planning Buffer Always add 2-3 months buffer to hiring plans. **Example:** If need engineer by July 1: - Start recruiting: April 1 (12 weeks) - Productivity: September 1 (2 months ramp) ## Budget Planning ### Compensation as % of Revenue **Early Stage (Seed):** - Total comp: 120-150% of revenue (burning cash to grow) - Engineering: 50-60% - Sales: 30-40% - Other: 20-30% **Growth Stage (Series A):** - Total comp: 70-100% of revenue - Engineering: 35-45% - Sales: 25-35% - Other: 20-30% ### Headcount Budget Formula ``` Total Comp Budget = Ī£ (Role Count Ɨ Fully-Loaded Cost Ɨ % of Year) Example: 3 Engineers Ɨ $202K Ɨ 100% = $606K 2 AEs Ɨ $230K Ɨ 75% (mid-year start) = $345K 1 PM Ɨ $162K Ɨ 100% = $162K Total: $1.1M ``` ## Additional Resources ### Reference Files - **`references/compensation-benchmarks.md`** - Detailed salary data by role, level, and location - **`references/equity-calculator.md`** - Equity sizing formulas and dilution scenarios ### Example Files - **`examples/seed-stage-hiring-plan.md`** - Complete hiring plan for seed-stage SaaS company - **`examples/org-chart-evolution.md`** - Organizational design from 5 to 50 people ## Quick Start To plan team composition: 1. **Identify stage** - Pre-seed, seed, or Series A 2. **Define roles** - What functions are needed now 3. **Prioritize hires** - Critical path for business goals 4. **Set compensation** - Base salary + equity by level 5. **Plan timeline** - Account for recruiting and ramp time 6. **Calculate budget** - Fully-loaded cost Ɨ headcount 7. **Design org chart** - Reporting structure and span of control 8. **Allocate equity** - Fair allocation that preserves pool For detailed compensation benchmarks and hiring plan templates, see `references/` and `examples/`.
šŸ‘0
šŸ‘ļø0
šŸ¤– Auto-discovered
šŸ¤–system prompt•7 months ago

market-sizing-analysis

This skill should be used when the user asks to "calculate TAM",

business
⭐1
# Market Sizing Analysis Comprehensive market sizing methodologies for calculating Total Addressable Market (TAM), Serviceable Available Market (SAM), and Serviceable Obtainable Market (SOM) for startup opportunities. ## Overview Market sizing provides the foundation for startup strategy, fundraising, and business planning. Calculate market opportunity using three complementary methodologies: top-down (industry reports), bottom-up (customer segment calculations), and value theory (willingness to pay). ## Core Concepts ### The Three-Tier Market Framework **TAM (Total Addressable Market)** - Total revenue opportunity if achieving 100% market share - Defines the universe of potential customers - Used for long-term vision and market validation - Example: All email marketing software revenue globally **SAM (Serviceable Available Market)** - Portion of TAM targetable with current product/service - Accounts for geographic, segment, or capability constraints - Represents realistic addressable opportunity - Example: AI-powered email marketing for e-commerce in North America **SOM (Serviceable Obtainable Market)** - Realistic market share achievable in 3-5 years - Accounts for competition, resources, and market dynamics - Used for financial projections and fundraising - Example: 2-5% of SAM based on competitive landscape ### When to Use Each Methodology **Top-Down Analysis** - Use when established market research exists - Best for mature, well-defined markets - Validates market existence and growth - Starts with industry reports and narrows down **Bottom-Up Analysis** - Use when targeting specific customer segments - Best for new or niche markets - Most credible for investors - Builds from customer data and pricing **Value Theory** - Use when creating new market categories - Best for disruptive innovations - Estimates based on value creation - Calculates willingness to pay for problem solution ## Three-Methodology Framework ### Methodology 1: Top-Down Analysis Start with total market size and narrow to addressable segments. **Process:** 1. Identify total market category from research reports 2. Apply geographic filters (target regions) 3. Apply segment filters (target industries/customers) 4. Calculate competitive positioning adjustments **Formula:** ``` TAM = Total Market Category Size SAM = TAM Ɨ Geographic % Ɨ Segment % SOM = SAM Ɨ Realistic Capture Rate (2-5%) ``` **When to use:** Established markets with available research (e.g., SaaS, fintech, e-commerce) **Strengths:** Quick, uses credible data, validates market existence **Limitations:** May overestimate for new categories, less granular ### Methodology 2: Bottom-Up Analysis Build market size from customer segment calculations. **Process:** 1. Define target customer segments 2. Estimate number of potential customers per segment 3. Determine average revenue per customer 4. Calculate realistic penetration rates **Formula:** ``` TAM = Ī£ (Segment Size Ɨ Annual Revenue per Customer) SAM = TAM Ɨ (Segments You Can Serve / Total Segments) SOM = SAM Ɨ Realistic Penetration Rate (Year 3-5) ``` **When to use:** B2B, niche markets, specific customer segments **Strengths:** Most credible for investors, granular, defensible **Limitations:** Requires detailed customer research, time-intensive ### Methodology 3: Value Theory Calculate based on value created and willingness to pay. **Process:** 1. Identify problem being solved 2. Quantify current cost of problem (time, money, inefficiency) 3. Calculate value of solution (savings, gains, efficiency) 4. Estimate willingness to pay (typically 10-30% of value) 5. Multiply by addressable customer base **Formula:** ``` Value per Customer = Problem Cost Ɨ % Solved by Solution Price per Customer = Value Ɨ Willingness to Pay % (10-30%) TAM = Total Potential Customers Ɨ Price per Customer SAM = TAM Ɨ % Meeting Buy Criteria SOM = SAM Ɨ Realistic Adoption Rate ``` **When to use:** New categories, disruptive innovations, unclear existing markets **Strengths:** Shows value creation, works for new markets **Limitations:** Requires assumptions, harder to validate ## Step-by-Step Process ### Step 1: Define the Market Clearly specify what market is being measured. **Questions to answer:** - What problem is being solved? - Who are the target customers? - What's the product/service category? - What's the geographic scope? - What's the time horizon? **Example:** - Problem: E-commerce companies struggle with email marketing automation - Customers: E-commerce stores with >$1M annual revenue - Category: AI-powered email marketing software - Geography: North America initially, global expansion - Horizon: 3-5 year opportunity ### Step 2: Gather Data Sources Identify credible data for calculations. **Top-Down Sources:** - Industry research reports (Gartner, Forrester, IDC) - Government statistics (Census, BLS, trade associations) - Public company filings and earnings - Market research firms (Statista, CB Insights, PitchBook) **Bottom-Up Sources:** - Customer interviews and surveys - Sales data and CRM records - Industry databases (LinkedIn, ZoomInfo, Crunchbase) - Competitive intelligence - Academic research **Value Theory Sources:** - Customer problem quantification - Time/cost studies - ROI case studies - Pricing research and willingness-to-pay surveys ### Step 3: Calculate TAM Apply chosen methodology to determine total market. **For Top-Down:** 1. Find total category size from research 2. Document data source and year 3. Apply growth rate if needed 4. Validate with multiple sources **For Bottom-Up:** 1. Count total potential customers 2. Calculate average annual revenue per customer 3. Multiply to get TAM 4. Break down by segment **For Value Theory:** 1. Quantify total addressable customer base 2. Calculate value per customer 3. Estimate pricing based on value 4. Multiply for TAM ### Step 4: Calculate SAM Narrow TAM to serviceable addressable market. **Apply Filters:** - Geographic constraints (regions you can serve) - Product limitations (features you currently have) - Customer requirements (size, industry, use case) - Distribution channel access - Regulatory or compliance restrictions **Formula:** ``` SAM = TAM Ɨ (% matching all filters) ``` **Example:** - TAM: $10B global email marketing - Geographic filter: 40% (North America) - Product filter: 30% (e-commerce focus) - Feature filter: 60% (need AI capabilities) - SAM = $10B Ɨ 0.40 Ɨ 0.30 Ɨ 0.60 = $720M ### Step 5: Calculate SOM Determine realistic obtainable market share. **Consider:** - Current market share of competitors - Typical market share for new entrants (2-5%) - Resources available (funding, team, time) - Go-to-market effectiveness - Competitive advantages - Time to achieve (3-5 years typically) **Conservative Approach:** ``` SOM (Year 3) = SAM Ɨ 2% SOM (Year 5) = SAM Ɨ 5% ``` **Example:** - SAM: $720M - Year 3 SOM: $720M Ɨ 2% = $14.4M - Year 5 SOM: $720M Ɨ 5% = $36M ### Step 6: Validate and Triangulate Cross-check using multiple methods. **Validation Techniques:** 1. Compare top-down and bottom-up results (should be within 30%) 2. Check against public company revenues in space 3. Validate customer count assumptions 4. Sense-check pricing assumptions 5. Review with industry experts 6. Compare to similar market categories **Red Flags:** - TAM that's too small (< $1B for VC-backed startups) - TAM that's too large (unsupported by data) - SOM that's too aggressive (> 10% in 5 years for new entrant) - Inconsistency between methodologies (> 50% difference) ## Industry-Specific Considerations ### SaaS Markets **Key Metrics:** - Number of potential businesses in target segment - Average contract value (ACV) - Typical market penetration rates - Expansion revenue potential **TAM Calculation:** ``` TAM = Total Target Companies Ɨ Average ACV Ɨ (1 + Expansion Rate) ``` ### Marketplace Markets **Key Metrics:** - Gross Merchandise Value (GMV) of category - Take rate (% of GMV you capture) - Total transactions or users **TAM Calculation:** ``` TAM = Total Category GMV Ɨ Expected Take Rate ``` ### Consumer Markets **Key Metrics:** - Total addressable users/households - Average revenue per user (ARPU) - Engagement frequency **TAM Calculation:** ``` TAM = Total Users Ɨ ARPU Ɨ Purchase Frequency per Year ``` ### B2B Services **Key Metrics:** - Number of target companies by size/industry - Average project value or retainer - Typical buying frequency **TAM Calculation:** ``` TAM = Total Target Companies Ɨ Average Deal Size Ɨ Deals per Year ``` ## Presenting Market Sizing ### For Investors **Structure:** 1. Market definition and problem scope 2. TAM/SAM/SOM with methodology 3. Data sources and assumptions 4. Growth projections and drivers 5. Competitive landscape context **Key Points:** - Lead with bottom-up calculation (most credible) - Show triangulation with top-down - Explain conservative assumptions - Link to revenue projections - Highlight market growth rate ### For Strategy **Structure:** 1. Addressable customer segments 2. Prioritization by opportunity size 3. Entry strategy by segment 4. Expected penetration timeline 5. Resource requirements **Key Points:** - Focus on SAM and SOM - Show segment-level detail - Connect to go-to-market plan - Identify expansion opportunities - Discuss competitive positioning ## Common Mistakes to Avoid **Mistake 1: Confusing TAM with SAM** - Don't claim entire market as addressable - Apply realistic product/geographic constraints - Be honest about serviceable market **Mistake 2: Overly Aggressive SOM** - New entrants rarely capture > 5% in 5 years - Account for competition and resources - Show realistic ramp timeline **Mistake 3: Using Only Top-Down** - Investors prefer bottom-up validation - Top-down alone lacks credibility - Always triangulate with multiple methods **Mistake 4: Cherry-Picking Data** - Use consistent, recent data sources - Don't mix methodologies inappropriately - Document all assumptions clearly **Mistake 5: Ignoring Market Dynamics** - Account for market growth/decline - Consider competitive intensity - Factor in switching costs and barriers ## Additional Resources ### Reference Files For detailed methodologies and frameworks: - **`references/methodology-deep-dive.md`** - Comprehensive guide to each methodology with step-by-step worksheets - **`references/data-sources.md`** - Curated list of market research sources, databases, and tools - **`references/industry-templates.md`** - Specific templates for SaaS, marketplace, consumer, B2B, and fintech markets ### Example Files Working examples with complete calculations: - **`examples/saas-market-sizing.md`** - Complete TAM/SAM/SOM for a B2B SaaS product - **`examples/marketplace-sizing.md`** - Marketplace platform market opportunity calculation - **`examples/value-theory-example.md`** - Value-based market sizing for disruptive innovation Use these examples as templates for your own market sizing analysis. Each includes real numbers, data sources, and assumptions documented clearly. ## Quick Start To perform market sizing analysis: 1. **Define the market** - Problem, customers, category, geography 2. **Choose methodology** - Bottom-up (preferred) or top-down + triangulation 3. **Gather data** - Industry reports, customer data, competitive intelligence 4. **Calculate TAM** - Apply methodology formula 5. **Narrow to SAM** - Apply product, geographic, segment filters 6. **Estimate SOM** - 2-5% realistic capture rate 7. **Validate** - Cross-check with alternative methods 8. **Document** - Show methodology, sources, assumptions 9. **Present** - Structure for audience (investors, strategy, operations) For detailed step-by-step guidance on each methodology, reference the files in `references/` directory. For complete worked examples, see `examples/` directory.
šŸ‘0
šŸ‘ļø0
šŸ¤– Auto-discovered
šŸ¤–system prompt•7 months ago

dotnet-backend-patterns

Master C#/.NET backend development patterns for building robust

coding
⭐1
# .NET Backend Development Patterns Master C#/.NET patterns for building production-grade APIs, MCP servers, and enterprise backends with modern best practices (2024/2025). ## When to Use This Skill - Developing new .NET Web APIs or MCP servers - Reviewing C# code for quality and performance - Designing service architectures with dependency injection - Implementing caching strategies with Redis - Writing unit and integration tests - Optimizing database access with EF Core or Dapper - Configuring applications with IOptions pattern - Handling errors and implementing resilience patterns ## Core Concepts ### 1. Project Structure (Clean Architecture) ``` src/ ā”œā”€ā”€ Domain/ # Core business logic (no dependencies) │ ā”œā”€ā”€ Entities/ │ ā”œā”€ā”€ Interfaces/ │ ā”œā”€ā”€ Exceptions/ │ └── ValueObjects/ ā”œā”€ā”€ Application/ # Use cases, DTOs, validation │ ā”œā”€ā”€ Services/ │ ā”œā”€ā”€ DTOs/ │ ā”œā”€ā”€ Validators/ │ └── Interfaces/ ā”œā”€ā”€ Infrastructure/ # External implementations │ ā”œā”€ā”€ Data/ # EF Core, Dapper repositories │ ā”œā”€ā”€ Caching/ # Redis, Memory cache │ ā”œā”€ā”€ External/ # HTTP clients, third-party APIs │ └── DependencyInjection/ # Service registration └── Api/ # Entry point ā”œā”€ā”€ Controllers/ # Or MinimalAPI endpoints ā”œā”€ā”€ Middleware/ ā”œā”€ā”€ Filters/ └── Program.cs ``` ### 2. Dependency Injection Patterns ```csharp // Service registration by lifetime public static class ServiceCollectionExtensions { public static IServiceCollection AddApplicationServices( this IServiceCollection services, IConfiguration configuration) { // Scoped: One instance per HTTP request services.AddScoped<IProductService, ProductService>(); services.AddScoped<IOrderService, OrderService>(); // Singleton: One instance for app lifetime services.AddSingleton<ICacheService, RedisCacheService>(); services.AddSingleton<IConnectionMultiplexer>(_ => ConnectionMultiplexer.Connect(configuration["Redis:Connection"]!)); // Transient: New instance every time services.AddTransient<IValidator<CreateOrderRequest>, CreateOrderValidator>(); // Options pattern for configuration services.Configure<CatalogOptions>(configuration.GetSection("Catalog")); services.Configure<RedisOptions>(configuration.GetSection("Redis")); // Factory pattern for conditional creation services.AddScoped<IPriceCalculator>(sp => { var options = sp.GetRequiredService<IOptions<PricingOptions>>().Value; return options.UseNewEngine ? sp.GetRequiredService<NewPriceCalculator>() : sp.GetRequiredService<LegacyPriceCalculator>(); }); // Keyed services (.NET 8+) services.AddKeyedScoped<IPaymentProcessor, StripeProcessor>("stripe"); services.AddKeyedScoped<IPaymentProcessor, PayPalProcessor>("paypal"); return services; } } // Usage with keyed services public class CheckoutService { public CheckoutService( [FromKeyedServices("stripe")] IPaymentProcessor stripeProcessor) { _processor = stripeProcessor; } } ``` ### 3. Async/Await Patterns ```csharp // āœ… CORRECT: Async all the way down public async Task<Product> GetProductAsync(string id, CancellationToken ct = default) { return await _repository.GetByIdAsync(id, ct); } // āœ… CORRECT: Parallel execution with WhenAll public async Task<(Stock, Price)> GetStockAndPriceAsync( string productId, CancellationToken ct = default) { var stockTask = _stockService.GetAsync(productId, ct); var priceTask = _priceService.GetAsync(productId, ct); await Task.WhenAll(stockTask, priceTask); return (await stockTask, await priceTask); } // āœ… CORRECT: ConfigureAwait in libraries public async Task<T> LibraryMethodAsync<T>(CancellationToken ct = default) { var result = await _httpClient.GetAsync(url, ct).ConfigureAwait(false); return await result.Content.ReadFromJsonAsync<T>(ct).ConfigureAwait(false); } // āœ… CORRECT: ValueTask for hot paths with caching public ValueTask<Product?> GetCachedProductAsync(string id) { if (_cache.TryGetValue(id, out Product? product)) return ValueTask.FromResult(product); return new ValueTask<Product?>(GetFromDatabaseAsync(id)); } // āŒ WRONG: Blocking on async (deadlock risk) var result = GetProductAsync(id).Result; // NEVER do this var result2 = GetProductAsync(id).GetAwaiter().GetResult(); // Also bad // āŒ WRONG: async void (except event handlers) public async void ProcessOrder() { } // Exceptions are lost // āŒ WRONG: Unnecessary Task.Run for already async code await Task.Run(async () => await GetDataAsync()); // Wastes thread ``` ### 4. Configuration with IOptions ```csharp // Configuration classes public class CatalogOptions { public const string SectionName = "Catalog"; public int DefaultPageSize { get; set; } = 50; public int MaxPageSize { get; set; } = 200; public TimeSpan CacheDuration { get; set; } = TimeSpan.FromMinutes(15); public bool EnableEnrichment { get; set; } = true; } public class RedisOptions { public const string SectionName = "Redis"; public string Connection { get; set; } = "localhost:6379"; public string KeyPrefix { get; set; } = "mcp:"; public int Database { get; set; } = 0; } // appsettings.json { "Catalog": { "DefaultPageSize": 50, "MaxPageSize": 200, "CacheDuration": "00:15:00", "EnableEnrichment": true }, "Redis": { "Connection": "localhost:6379", "KeyPrefix": "mcp:", "Database": 0 } } // Registration services.Configure<CatalogOptions>(configuration.GetSection(CatalogOptions.SectionName)); services.Configure<RedisOptions>(configuration.GetSection(RedisOptions.SectionName)); // Usage with IOptions (singleton, read once at startup) public class CatalogService { private readonly CatalogOptions _options; public CatalogService(IOptions<CatalogOptions> options) { _options = options.Value; } } // Usage with IOptionsSnapshot (scoped, re-reads on each request) public class DynamicService { private readonly CatalogOptions _options; public DynamicService(IOptionsSnapshot<CatalogOptions> options) { _options = options.Value; // Fresh value per request } } // Usage with IOptionsMonitor (singleton, notified on changes) public class MonitoredService { private CatalogOptions _options; public MonitoredService(IOptionsMonitor<CatalogOptions> monitor) { _options = monitor.CurrentValue; monitor.OnChange(newOptions => _options = newOptions); } } ``` ### 5. Result Pattern (Avoiding Exceptions for Flow Control) ```csharp // Generic Result type public class Result<T> { public bool IsSuccess { get; } public T? Value { get; } public string? Error { get; } public string? ErrorCode { get; } private Result(bool isSuccess, T? value, string? error, string? errorCode) { IsSuccess = isSuccess; Value = value; Error = error; ErrorCode = errorCode; } public static Result<T> Success(T value) => new(true, value, null, null); public static Result<T> Failure(string error, string? code = null) => new(false, default, error, code); public Result<TNew> Map<TNew>(Func<T, TNew> mapper) => IsSuccess ? Result<TNew>.Success(mapper(Value!)) : Result<TNew>.Failure(Error!, ErrorCode); public async Task<Result<TNew>> MapAsync<TNew>(Func<T, Task<TNew>> mapper) => IsSuccess ? Result<TNew>.Success(await mapper(Value!)) : Result<TNew>.Failure(Error!, ErrorCode); } // Usage in service public async Task<Result<Order>> CreateOrderAsync(CreateOrderRequest request, CancellationToken ct) { // Validation var validation = await _validator.ValidateAsync(request, ct); if (!validation.IsValid) return Result<Order>.Failure( validation.Errors.First().ErrorMessage, "VALIDATION_ERROR"); // Business rule check var stock = await _stockService.CheckAsync(request.ProductId, request.Quantity, ct); if (!stock.IsAvailable) return Result<Order>.Failure( $"Insufficient stock: {stock.Available} available, {request.Quantity} requested", "INSUFFICIENT_STOCK"); // Create order var order = await _repository.CreateAsync(request.ToEntity(), ct); return Result<Order>.Success(order); } // Usage in controller/endpoint app.MapPost("/orders", async ( CreateOrderRequest request, IOrderService orderService, CancellationToken ct) => { var result = await orderService.CreateOrderAsync(request, ct); return result.IsSuccess ? Results.Created($"/orders/{result.Value!.Id}", result.Value) : Results.BadRequest(new { error = result.Error, code = result.ErrorCode }); }); ``` ## Data Access Patterns ### Entity Framework Core ```csharp // DbContext configuration public class AppDbContext : DbContext { public DbSet<Product> Products => Set<Product>(); public DbSet<Order> Orders => Set<Order>(); protected override void OnModelCreating(ModelBuilder modelBuilder) { // Apply all configurations from assembly modelBuilder.ApplyConfigurationsFromAssembly(typeof(AppDbContext).Assembly); // Global query filters modelBuilder.Entity<Product>().HasQueryFilter(p => !p.IsDeleted); } } // Entity configuration public class ProductConfiguration : IEntityTypeConfiguration<Product> { public void Configure(EntityTypeBuilder<Product> builder) { builder.ToTable("Products"); builder.HasKey(p => p.Id); builder.Property(p => p.Id).HasMaxLength(40); builder.Property(p => p.Name).HasMaxLength(200).IsRequired(); builder.Property(p => p.Price).HasPrecision(18, 2); builder.HasIndex(p => p.Sku).IsUnique(); builder.HasIndex(p => new { p.CategoryId, p.Name }); builder.HasMany(p => p.OrderItems) .WithOne(oi => oi.Product) .HasForeignKey(oi => oi.ProductId); } } // Repository with EF Core public class ProductRepository : IProductRepository { private readonly AppDbContext _context; public async Task<Product?> GetByIdAsync(string id, CancellationToken ct = default) { return await _context.Products .AsNoTracking() .FirstOrDefaultAsync(p => p.Id == id, ct); } public async Task<IReadOnlyList<Product>> SearchAsync( ProductSearchCriteria criteria, CancellationToken ct = default) { var query = _context.Products.AsNoTracking(); if (!string.IsNullOrWhiteSpace(criteria.SearchTerm)) query = query.Where(p => EF.Functions.Like(p.Name, $"%{criteria.SearchTerm}%")); if (criteria.CategoryId.HasValue) query = query.Where(p => p.CategoryId == criteria.CategoryId); if (criteria.MinPrice.HasValue) query = query.Where(p => p.Price >= criteria.MinPrice); if (criteria.MaxPrice.HasValue) query = query.Where(p => p.Price <= criteria.MaxPrice); return await query .OrderBy(p => p.Name) .Skip((criteria.Page - 1) * criteria.PageSize) .Take(criteria.PageSize) .ToListAsync(ct); } } ``` ### Dapper for Performance ```csharp public class DapperProductRepository : IProductRepository { private readonly IDbConnection _connection; public async Task<Product?> GetByIdAsync(string id, CancellationToken ct = default) { const string sql = """ SELECT Id, Name, Sku, Price, CategoryId, Stock, CreatedAt FROM Products WHERE Id = @Id AND IsDeleted = 0 """; return await _connection.QueryFirstOrDefaultAsync<Product>( new CommandDefinition(sql, new { Id = id }, cancellationToken: ct)); } public async Task<IReadOnlyList<Product>> SearchAsync( ProductSearchCriteria criteria, CancellationToken ct = default) { var sql = new StringBuilder(""" SELECT Id, Name, Sku, Price, CategoryId, Stock, CreatedAt FROM Products WHERE IsDeleted = 0 """); var parameters = new DynamicParameters(); if (!string.IsNullOrWhiteSpace(criteria.SearchTerm)) { sql.Append(" AND Name LIKE @SearchTerm"); parameters.Add("SearchTerm", $"%{criteria.SearchTerm}%"); } if (criteria.CategoryId.HasValue) { sql.Append(" AND CategoryId = @CategoryId"); parameters.Add("CategoryId", criteria.CategoryId); } if (criteria.MinPrice.HasValue) { sql.Append(" AND Price >= @MinPrice"); parameters.Add("MinPrice", criteria.MinPrice); } if (criteria.MaxPrice.HasValue) { sql.Append(" AND Price <= @MaxPrice"); parameters.Add("MaxPrice", criteria.MaxPrice); } sql.Append(" ORDER BY Name OFFSET @Offset ROWS FETCH NEXT @PageSize ROWS ONLY"); parameters.Add("Offset", (criteria.Page - 1) * criteria.PageSize); parameters.Add("PageSize", criteria.PageSize); var results = await _connection.QueryAsync<Product>( new CommandDefinition(sql.ToString(), parameters, cancellationToken: ct)); return results.ToList(); } // Multi-mapping for related data public async Task<Order?> GetOrderWithItemsAsync(int orderId, CancellationToken ct = default) { const string sql = """ SELECT o.*, oi.*, p.* FROM Orders o LEFT JOIN OrderItems oi ON o.Id = oi.OrderId LEFT JOIN Products p ON oi.ProductId = p.Id WHERE o.Id = @OrderId """; var orderDictionary = new Dictionary<int, Order>(); await _connection.QueryAsync<Order, OrderItem, Product, Order>( new CommandDefinition(sql, new { OrderId = orderId }, cancellationToken: ct), (order, item, product) => { if (!orderDictionary.TryGetValue(order.Id, out var existingOrder)) { existingOrder = order; existingOrder.Items = new List<OrderItem>(); orderDictionary.Add(order.Id, existingOrder); } if (item != null) { item.Product = product; existingOrder.Items.Add(item); } return existingOrder; }, splitOn: "Id,Id"); return orderDictionary.Values.FirstOrDefault(); } } ``` ## Caching Patterns ### Multi-Level Cache with Redis ```csharp public class CachedProductService : IProductService { private readonly IProductRepository _repository; private readonly IMemoryCache _memoryCache; private readonly IDistributedCache _distributedCache; private readonly ILogger<CachedProductService> _logger; private static readonly TimeSpan MemoryCacheDuration = TimeSpan.FromMinutes(1); private static readonly TimeSpan DistributedCacheDuration = TimeSpan.FromMinutes(15); public async Task<Product?> GetByIdAsync(string id, CancellationToken ct = default) { var cacheKey = $"product:{id}"; // L1: Memory cache (in-process, fastest) if (_memoryCache.TryGetValue(cacheKey, out Product? cached)) { _logger.LogDebug("L1 cache hit for {CacheKey}", cacheKey); return cached; } // L2: Distributed cache (Redis) var distributed = await _distributedCache.GetStringAsync(cacheKey, ct); if (distributed != null) { _logger.LogDebug("L2 cache hit for {CacheKey}", cacheKey); var product = JsonSerializer.Deserialize<Product>(distributed); // Populate L1 _memoryCache.Set(cacheKey, product, MemoryCacheDuration); return product; } // L3: Database _logger.LogDebug("Cache miss for {CacheKey}, fetching from database", cacheKey); var fromDb = await _repository.GetByIdAsync(id, ct); if (fromDb != null) { var serialized = JsonSerializer.Serialize(fromDb); // Populate both caches await _distributedCache.SetStringAsync( cacheKey, serialized, new DistributedCacheEntryOptions { AbsoluteExpirationRelativeToNow = DistributedCacheDuration }, ct); _memoryCache.Set(cacheKey, fromDb, MemoryCacheDuration); } return fromDb; } public async Task InvalidateAsync(string id, CancellationToken ct = default) { var cacheKey = $"product:{id}"; _memoryCache.Remove(cacheKey); await _distributedCache.RemoveAsync(cacheKey, ct); _logger.LogInformation("Invalidated cache for {CacheKey}", cacheKey); } } // Stale-while-revalidate pattern public class StaleWhileRevalidateCache<T> { private readonly IDistributedCache _cache; private readonly TimeSpan _freshDuration; private readonly TimeSpan _staleDuration; public async Task<T?> GetOrCreateAsync( string key, Func<CancellationToken, Task<T>> factory, CancellationToken ct = default) { var cached = await _cache.GetStringAsync(key, ct); if (cached != null) { var entry = JsonSerializer.Deserialize<CacheEntry<T>>(cached)!; if (entry.IsStale && !entry.IsExpired) { // Return stale data immediately, refresh in background _ = Task.Run(async () => { var fresh = await factory(CancellationToken.None); await SetAsync(key, fresh, CancellationToken.None); }); } if (!entry.IsExpired) return entry.Value; } // Cache miss or expired var value = await factory(ct); await SetAsync(key, value, ct); return value; } private record CacheEntry<TValue>(TValue Value, DateTime CreatedAt) { public bool IsStale => DateTime.UtcNow - CreatedAt > _freshDuration; public bool IsExpired => DateTime.UtcNow - CreatedAt > _staleDuration; } } ``` ## Testing Patterns ### Unit Tests with xUnit and Moq ```csharp public class OrderServiceTests { private readonly Mock<IOrderRepository> _mockRepository; private readonly Mock<IStockService> _mockStockService; private readonly Mock<IValidator<CreateOrderRequest>> _mockValidator; private readonly OrderService _sut; // System Under Test public OrderServiceTests() { _mockRepository = new Mock<IOrderRepository>(); _mockStockService = new Mock<IStockService>(); _mockValidator = new Mock<IValidator<CreateOrderRequest>>(); // Default: validation passes _mockValidator .Setup(v => v.ValidateAsync(It.IsAny<CreateOrderRequest>(), It.IsAny<
šŸ‘0
šŸ‘ļø0
šŸ¤– Auto-discovered
šŸ¤–system prompt•7 months ago

python-configuration

Python configuration management via environment variables and typed

coding
⭐1
# Python Configuration Management Externalize configuration from code using environment variables and typed settings. Well-managed configuration enables the same code to run in any environment without modification. ## When to Use This Skill - Setting up a new project's configuration system - Migrating from hardcoded values to environment variables - Implementing pydantic-settings for typed configuration - Managing secrets and sensitive values - Creating environment-specific settings (dev/staging/prod) - Validating configuration at application startup ## Core Concepts ### 1. Externalized Configuration All environment-specific values (URLs, secrets, feature flags) come from environment variables, not code. ### 2. Typed Settings Parse and validate configuration into typed objects at startup, not scattered throughout code. ### 3. Fail Fast Validate all required configuration at application boot. Missing config should crash immediately with a clear message. ### 4. Sensible Defaults Provide reasonable defaults for local development while requiring explicit values for sensitive settings. ## Quick Start ```python from pydantic_settings import BaseSettings from pydantic import Field class Settings(BaseSettings): database_url: str = Field(alias="DATABASE_URL") api_key: str = Field(alias="API_KEY") debug: bool = Field(default=False, alias="DEBUG") settings = Settings() # Loads from environment ``` ## Fundamental Patterns ### Pattern 1: Typed Settings with Pydantic Create a central settings class that loads and validates all configuration. ```python from pydantic_settings import BaseSettings from pydantic import Field, PostgresDsn, ValidationError import sys class Settings(BaseSettings): """Application configuration loaded from environment variables.""" # Database db_host: str = Field(alias="DB_HOST") db_port: int = Field(default=5432, alias="DB_PORT") db_name: str = Field(alias="DB_NAME") db_user: str = Field(alias="DB_USER") db_password: str = Field(alias="DB_PASSWORD") # Redis redis_url: str = Field(default="redis://localhost:6379", alias="REDIS_URL") # API Keys api_secret_key: str = Field(alias="API_SECRET_KEY") # Feature flags enable_new_feature: bool = Field(default=False, alias="ENABLE_NEW_FEATURE") model_config = { "env_file": ".env", "env_file_encoding": "utf-8", } # Create singleton instance at module load try: settings = Settings() except ValidationError as e: print(f"Configuration error:\n{e}") sys.exit(1) ``` Import `settings` throughout your application: ```python from myapp.config import settings def get_database_connection(): return connect( host=settings.db_host, port=settings.db_port, database=settings.db_name, ) ``` ### Pattern 2: Fail Fast on Missing Configuration Required settings should crash the application immediately with a clear error. ```python from pydantic_settings import BaseSettings from pydantic import Field, ValidationError import sys class Settings(BaseSettings): # Required - no default means it must be set api_key: str = Field(alias="API_KEY") database_url: str = Field(alias="DATABASE_URL") # Optional with defaults log_level: str = Field(default="INFO", alias="LOG_LEVEL") try: settings = Settings() except ValidationError as e: print("=" * 60) print("CONFIGURATION ERROR") print("=" * 60) for error in e.errors(): field = error["loc"][0] print(f" - {field}: {error['msg']}") print("\nPlease set the required environment variables.") sys.exit(1) ``` A clear error at startup is better than a cryptic `None` failure mid-request. ### Pattern 3: Local Development Defaults Provide sensible defaults for local development while requiring explicit values for secrets. ```python class Settings(BaseSettings): # Has local default, but prod will override db_host: str = Field(default="localhost", alias="DB_HOST") db_port: int = Field(default=5432, alias="DB_PORT") # Always required - no default for secrets db_password: str = Field(alias="DB_PASSWORD") api_secret_key: str = Field(alias="API_SECRET_KEY") # Development convenience debug: bool = Field(default=False, alias="DEBUG") model_config = {"env_file": ".env"} ``` Create a `.env` file for local development (never commit this): ```bash # .env (add to .gitignore) DB_PASSWORD=local_dev_password API_SECRET_KEY=dev-secret-key DEBUG=true ``` ### Pattern 4: Namespaced Environment Variables Prefix related variables for clarity and easy debugging. ```bash # Database configuration DB_HOST=localhost DB_PORT=5432 DB_NAME=myapp DB_USER=admin DB_PASSWORD=secret # Redis configuration REDIS_URL=redis://localhost:6379 REDIS_MAX_CONNECTIONS=10 # Authentication AUTH_SECRET_KEY=your-secret-key AUTH_TOKEN_EXPIRY_SECONDS=3600 AUTH_ALGORITHM=HS256 # Feature flags FEATURE_NEW_CHECKOUT=true FEATURE_BETA_UI=false ``` Makes `env | grep DB_` useful for debugging. ## Advanced Patterns ### Pattern 5: Type Coercion Pydantic handles common conversions automatically. ```python from pydantic_settings import BaseSettings from pydantic import Field, field_validator class Settings(BaseSettings): # Automatically converts "true", "1", "yes" to True debug: bool = False # Automatically converts string to int max_connections: int = 100 # Parse comma-separated string to list allowed_hosts: list[str] = Field(default_factory=list) @field_validator("allowed_hosts", mode="before") @classmethod def parse_allowed_hosts(cls, v: str | list[str]) -> list[str]: if isinstance(v, str): return [host.strip() for host in v.split(",") if host.strip()] return v ``` Usage: ```bash ALLOWED_HOSTS=example.com,api.example.com,localhost MAX_CONNECTIONS=50 DEBUG=true ``` ### Pattern 6: Environment-Specific Configuration Use an environment enum to switch behavior. ```python from enum import Enum from pydantic_settings import BaseSettings from pydantic import Field, computed_field class Environment(str, Enum): LOCAL = "local" STAGING = "staging" PRODUCTION = "production" class Settings(BaseSettings): environment: Environment = Field( default=Environment.LOCAL, alias="ENVIRONMENT", ) # Settings that vary by environment log_level: str = Field(default="DEBUG", alias="LOG_LEVEL") @computed_field @property def is_production(self) -> bool: return self.environment == Environment.PRODUCTION @computed_field @property def is_local(self) -> bool: return self.environment == Environment.LOCAL # Usage if settings.is_production: configure_production_logging() else: configure_debug_logging() ``` ### Pattern 7: Nested Configuration Groups Organize related settings into nested models. ```python from pydantic import BaseModel from pydantic_settings import BaseSettings class DatabaseSettings(BaseModel): host: str = "localhost" port: int = 5432 name: str user: str password: str class RedisSettings(BaseModel): url: str = "redis://localhost:6379" max_connections: int = 10 class Settings(BaseSettings): database: DatabaseSettings redis: RedisSettings debug: bool = False model_config = { "env_nested_delimiter": "__", "env_file": ".env", } ``` Environment variables use double underscore for nesting: ```bash DATABASE__HOST=db.example.com DATABASE__PORT=5432 DATABASE__NAME=myapp DATABASE__USER=admin DATABASE__PASSWORD=secret REDIS__URL=redis://redis.example.com:6379 ``` ### Pattern 8: Secrets from Files For container environments, read secrets from mounted files. ```python from pydantic_settings import BaseSettings from pydantic import Field from pathlib import Path class Settings(BaseSettings): # Read from environment variable or file db_password: str = Field(alias="DB_PASSWORD") model_config = { "secrets_dir": "/run/secrets", # Docker secrets location } ``` Pydantic will look for `/run/secrets/db_password` if the env var isn't set. ### Pattern 9: Configuration Validation Add custom validation for complex requirements. ```python from pydantic_settings import BaseSettings from pydantic import Field, model_validator class Settings(BaseSettings): db_host: str = Field(alias="DB_HOST") db_port: int = Field(alias="DB_PORT") read_replica_host: str | None = Field(default=None, alias="READ_REPLICA_HOST") read_replica_port: int = Field(default=5432, alias="READ_REPLICA_PORT") @model_validator(mode="after") def validate_replica_settings(self): if self.read_replica_host and self.read_replica_port == self.db_port: if self.read_replica_host == self.db_host: raise ValueError( "Read replica cannot be the same as primary database" ) return self ``` ## Best Practices Summary 1. **Never hardcode config** - All environment-specific values from env vars 2. **Use typed settings** - Pydantic-settings with validation 3. **Fail fast** - Crash on missing required config at startup 4. **Provide dev defaults** - Make local development easy 5. **Never commit secrets** - Use `.env` files (gitignored) or secret managers 6. **Namespace variables** - `DB_HOST`, `REDIS_URL` for clarity 7. **Import settings singleton** - Don't call `os.getenv()` throughout code 8. **Document all variables** - README should list required env vars 9. **Validate early** - Check config correctness at boot time 10. **Use secrets_dir** - Support mounted secrets in containers
šŸ‘0
šŸ‘ļø0
šŸ¤– Auto-discovered