Skip to main content
EVOKORE// BROWSE
>

./browse/prompts

25 NODES
πŸ€–system promptβ€’6 months ago

Multi-Server MCP Aggregation Pattern

Aggregate tools across multiple MCP child servers with prefixing, collision avoidance, and routing rules that stay deterministic.

architecture
⭐1
# Multi-Server MCP Aggregation Pattern Imported from curated first-party documentation sources. ## What this covers Use this pattern when an MCP host must broker tools from multiple child servers without sacrificing clarity or control. ## Use this when - Combining tools from several MCP backends - Avoiding tool-name collisions across providers - Keeping origin and routing visible during execution ## Expected outcomes - Server-prefixed names make tool origins obvious - Collisions are avoided without brittle manual renaming - Operators can extend the tool surface without losing determinism ## Source synthesis - EVOKORE-MCP/docs/AGENT33_IMPROVEMENT_INSTRUCTIONS.md (https://github.com/mattmre/EVOKORE-MCP/blob/main/docs/AGENT33_IMPROVEMENT_INSTRUCTIONS.md) - EVOKORE-MCP/docs/TOOLS_AND_DISCOVERY.md (https://github.com/mattmre/EVOKORE-MCP/blob/main/docs/TOOLS_AND_DISCOVERY.md) ## Dedupe notes Uses the improvement-transfer doc as the primary source, with the discovery doc covering prefixing and compatibility details. ## Source excerpts ### EVOKORE-MCP/docs/AGENT33_IMPROVEMENT_INSTRUCTIONS.md > **Purpose**: Feed this file into Claude Code CLI when working on the Agent33 repo. It contains patterns, architectures, and capabilities proven in EVOKORE-MCP that Agent33 should adopt. --- ## 1. Multi-Server MCP Aggregation Pattern **What Agent33 lacks**: Agent33's MCP server (Phase 43) is a single-endpoint bridge. It doesn't aggregate multiple child MCP servers behind a unified namespace. **What to build**: A proxy layer that spawns and manages multiple child MCP servers from a single config file, presenting them as one unified tool surface. ### Implementation spec: ``` mcp.config.json { "servers": { "github": { "command": "npx", "args": ["-y", "@modelcontextprotocol/server-github"], "env": { "GITHUB_TOKEN": "${GITHUB_TOKEN}" } }, "fs": { "command": "npx", "args": ["-y", "@modelcontextprotocol/server-filesystem", "./"] }, "elevenlabs": { "command": "uvx", "args": ["elevenlabs-mcp"], "env": { "ELEVENLABS_API_KEY": "${ELEVENLABS_API_KEY}" } } } } ``` **Key patterns from EVOKORE**: - **Tool name prefixing**: Every proxied tool gets renamed `{serverId}_{originalName}` to prevent namespace collisions (e.g., `github_create_issue`, `fs_read_file`). First-registration-wins for duplicates. - **Environment interpolation**: `${VAR}` syntax in `env` blocks resolved ... ### EVOKORE-MCP/docs/TOOLS_AND_DISCOVERY.md This page explains how EVOKORE presents tools, how proxy names are built, and how `discover_tools` changes the visible tool surface. ## Two tool populations ### Native EVOKORE tools These tools are defined by EVOKORE itself: - `docs_architect` - `skill_creator` - `resolve_workflow` - `search_skills` - `get_skill_help` - `discover_tools` Properties: - always available - always visible - not subject to proxy prefixing ### Proxied child-server tools These come from child servers in `mcp.config.json`. Current configured sources: - `github` - `fs` - optional `elevenlabs` Properties: - fetched from child servers at startup - renamed with server prefixes - governed by `permissions.yml` - routed through `ProxyManager` ## Prefixing and compatibility EVOKORE rewrites proxied tool names to: ```text ${serverId}_${tool.name} ``` Why this exists: - prevents tool-name collisions across child servers - makes origin obvious during execution and review - keeps exact-name routing deterministic Examples: | Upstream tool | EVOKORE-exposed tool | |---|---| | `read_file` from `fs` | `fs_read_file` | | `create_issue` from `github` | `github_create_issue` | ### Duplicate-prefixed name policy If two child registrations would create the same final prefixed name: - the first registrati ...
πŸ‘0
πŸ‘οΈ0
docs
πŸ€–system promptβ€’6 months ago

Dynamic Tool Discovery and Session Activation

Expose only native tools at session start, then activate proxied tools on demand through a searchable discovery layer.

architecture
⭐1
# Dynamic Tool Discovery and Session Activation Imported from curated first-party documentation sources. ## What this covers Use this skill when you need smaller tool payloads, exact-name compatibility, and session-scoped tool activation. ## Use this when - Reducing tool-list bloat for focused sessions - Preserving exact-name tool calls while hiding noise - Activating proxied tools only when a task requires them ## Expected outcomes - Discovery changes listing visibility without breaking routing - Session-scoped activation keeps tools relevant to the task - Operators can trade compatibility against payload size intentionally ## Source synthesis - EVOKORE-MCP/docs/TOOLS_AND_DISCOVERY.md (https://github.com/mattmre/EVOKORE-MCP/blob/main/docs/TOOLS_AND_DISCOVERY.md) ## Dedupe notes Uses the canonical EVOKORE-MCP discovery doc instead of duplicating the same concept from walkthrough and migration docs. ## Source excerpts ### EVOKORE-MCP/docs/TOOLS_AND_DISCOVERY.md This page explains how EVOKORE presents tools, how proxy names are built, and how `discover_tools` changes the visible tool surface. ## Two tool populations ### Native EVOKORE tools These tools are defined by EVOKORE itself: - `docs_architect` - `skill_creator` - `resolve_workflow` - `search_skills` - `get_skill_help` - `discover_tools` Properties: - always available - always visible - not subject to proxy prefixing ### Proxied child-server tools These come from child servers in `mcp.config.json`. Current configured sources: - `github` - `fs` - optional `elevenlabs` Properties: - fetched from child servers at startup - renamed with server prefixes - governed by `permissions.yml` - routed through `ProxyManager` ## Prefixing and compatibility EVOKORE rewrites proxied tool names to: ```text ${serverId}_${tool.name} ``` Why this exists: - prevents tool-name collisions across child servers - makes origin obvious during execution and review - keeps exact-name routing deterministic Examples: | Upstream tool | EVOKORE-exposed tool | |---|---| | `read_file` from `fs` | `fs_read_file` | | `create_issue` from `github` | `github_create_issue` | ### Duplicate-prefixed name policy If two child registrations would create the same final prefixed name: - the first registration wins - later duplicates are skipped - EVOKORE logs a warning and duplicate summary ## Discovery modes | Mode | What `tools/list` returns | Best for | |---|---|---| | `legacy` | all native + proxied tools | maximum compatibility | | `dynamic` | native tools + session-activated proxied tools | smaller initial tool payloads | Environment toggle: ```bash EVOKORE_TOOL_DISCOVERY_MODE=legacy EVOKORE_TOOL_DISCOVERY_MODE=dynamic ``` ## Dynamic discovery lifecycle In `dynamic` mode, EVOKORE uses a session-scoped activation set. Lifecycle: 1. session starts with only native tools visible 2. user/model calls `discover_tools` 3. `ToolCatalogIndex` searches the merged native + proxied catalog 4. matching proxied tools are activated for that session 5. EVOKORE emits `sendToolListChanged()` best-effort 6. client re-runs `tools/list` or auto-refreshes ```mermaid flowchart TD A[Session starts in dynamic mode] --> B[tools/list shows native tools] B --> C[discover_tools query] C --> D[ToolCatalogIndex searches merged catalog] D --> E[Matching proxied tools added to session activation set] E --> F[sendToolListChanged best-effort] F --> G[Client refreshes tools/list] G --> H[Activated proxied tools now visible] D --> I[Exact-name proxied call still works eve ...
πŸ‘0
πŸ‘οΈ0
docs
πŸ“textβ€’6 months ago

Docker-Backed Jupyter Kernel Operator

Operate containerized Jupyter kernels with enablement, smoke checks, cleanup, and failure handling baked into the runbook.

devops
⭐1
# Docker-Backed Jupyter Kernel Operator Imported from curated first-party documentation sources. ## What this covers Use this runbook when you need isolated notebook execution, clear container lifecycle management, and troubleshooting guidance. ## Use this when - Enabling containerized notebook workflows - Cleaning up kernel containers safely - Troubleshooting notebook execution environments ## Expected outcomes - Kernel startup and cleanup steps are documented end-to-end - Failure modes are easier to diagnose - Notebook execution can be repeated without manual guesswork ## Source synthesis - AGENT33/docs/runbooks/jupyter-kernel-containers.md (https://github.com/mattmre/AGENT33/blob/main/docs/runbooks/jupyter-kernel-containers.md) ## Dedupe notes Uses the AGENT33 Jupyter container runbook as a focused operations import without duplicating broader walkthroughs. ## Source excerpts ### AGENT33/docs/runbooks/jupyter-kernel-containers.md ## Purpose Operate the Docker-backed Jupyter kernel adapter introduced for Phase 38 Stage 3 / Phase 42 follow-on work. ## Enablement Set: - `JUPYTER_KERNEL_ENABLED=true` - `JUPYTER_KERNEL_MODE=docker` Optional settings: - `JUPYTER_KERNEL_DOCKER_IMAGE` - `JUPYTER_KERNEL_ALLOWED_IMAGES` - `JUPYTER_KERNEL_NETWORK_ENABLED` - `JUPYTER_KERNEL_MOUNT_WORKDIR` - `JUPYTER_KERNEL_CONTAINER_WORKDIR` ## Operational Notes - Docker mode publishes kernel ports to the host and mounts a per-session runtime directory containing the Jupyter connection file. - When `JUPYTER_KERNEL_NETWORK_ENABLED=false`, the adapter starts containers with `--network none`. - Working-directory mounting is opt-in and should only point at paths already approved by workflow / execution policy. - The adapter enforces an image allowlist when one is configured. ## Failure Modes - `jupyter_client not installed`: install with `pip install agent33[jupyter]` - `docker executable not found`: install Docker and ensure `docker` is on `PATH` - `Docker image ... is not permitted`: align the requested image with `JUPYTER_KERNEL_ALLOWED_IMAGES` - kernel startup timeout: inspect Docker logs for the session container and verify the image includes `ipykernel` ## Cleanup - One-shot sessions are removed after execution. - Stateful sessions are removed explicitly or via adapter shutdown. - Forced cleanup uses `docker rm -f <container>` and deletes the runtime connection directory. ## Quick Smoke Workflow Register a minimal workflow that exercises the Docker-backed `code-interpreter` tool: ```bash curl -X POST http://localhost:8000/v1/workflows/ \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{ "name": "docker-kernel-smoke", "version": "1.0.0", "description": "Validate Docker-backed Jupyter execution", "triggers": {"manual": true}, "inputs": {}, "outputs": { "result": {"type": "object"} }, "steps": [ { "id": "run-notebook-code", "action": "execute-code", "inputs": { "tool_id": "code-interpreter", "language": "python", "code": "print(6 * 7)" } } ], "execution": {"mode": "sequential"} }' ``` Then execute it: ```bash curl -X POST http://localhost:8000/v1/workflows/docker-kernel-smoke/execute \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{"inputs": {}}' ```
πŸ‘0
πŸ‘οΈ0
docs
πŸ€–system promptβ€’7 months ago

cost-optimization

Optimize cloud costs through resource rightsizing, tagging

architecture
⭐1
# Cloud Cost Optimization Strategies and patterns for optimizing cloud costs across AWS, Azure, and GCP. ## Purpose Implement systematic cost optimization strategies to reduce cloud spending while maintaining performance and reliability. ## When to Use - Reduce cloud spending - Right-size resources - Implement cost governance - Optimize multi-cloud costs - Meet budget constraints ## Cost Optimization Framework ### 1. Visibility - Implement cost allocation tags - Use cloud cost management tools - Set up budget alerts - Create cost dashboards ### 2. Right-Sizing - Analyze resource utilization - Downsize over-provisioned resources - Use auto-scaling - Remove idle resources ### 3. Pricing Models - Use reserved capacity - Leverage spot/preemptible instances - Implement savings plans - Use committed use discounts ### 4. Architecture Optimization - Use managed services - Implement caching - Optimize data transfer - Use lifecycle policies ## AWS Cost Optimization ### Reserved Instances ``` Savings: 30-72% vs On-Demand Term: 1 or 3 years Payment: All/Partial/No upfront Flexibility: Standard or Convertible ``` ### Savings Plans ``` Compute Savings Plans: 66% savings EC2 Instance Savings Plans: 72% savings Applies to: EC2, Fargate, Lambda Flexible across: Instance families, regions, OS ``` ### Spot Instances ``` Savings: Up to 90% vs On-Demand Best for: Batch jobs, CI/CD, stateless workloads Risk: 2-minute interruption notice Strategy: Mix with On-Demand for resilience ``` ### S3 Cost Optimization ```hcl resource "aws_s3_bucket_lifecycle_configuration" "example" { bucket = aws_s3_bucket.example.id rule { id = "transition-to-ia" status = "Enabled" transition { days = 30 storage_class = "STANDARD_IA" } transition { days = 90 storage_class = "GLACIER" } expiration { days = 365 } } } ``` ## Azure Cost Optimization ### Reserved VM Instances - 1 or 3 year terms - Up to 72% savings - Flexible sizing - Exchangeable ### Azure Hybrid Benefit - Use existing Windows Server licenses - Up to 80% savings with RI - Available for Windows and SQL Server ### Azure Advisor Recommendations - Right-size VMs - Delete unused resources - Use reserved capacity - Optimize storage ## GCP Cost Optimization ### Committed Use Discounts - 1 or 3 year commitment - Up to 57% savings - Applies to vCPUs and memory - Resource-based or spend-based ### Sustained Use Discounts - Automatic discounts - Up to 30% for running instances - No commitment required - Applies to Compute Engine, GKE ### Preemptible VMs - Up to 80% savings - 24-hour maximum runtime - Best for batch workloads ## Tagging Strategy ### AWS Tagging ```hcl locals { common_tags = { Environment = "production" Project = "my-project" CostCenter = "engineering" Owner = "team@example.com" ManagedBy = "terraform" } } resource "aws_instance" "example" { ami = "ami-12345678" instance_type = "t3.medium" tags = merge( local.common_tags, { Name = "web-server" } ) } ``` **Reference:** See `references/tagging-standards.md` ## Cost Monitoring ### Budget Alerts ```hcl # AWS Budget resource "aws_budgets_budget" "monthly" { name = "monthly-budget" budget_type = "COST" limit_amount = "1000" limit_unit = "USD" time_period_start = "2024-01-01_00:00" time_unit = "MONTHLY" notification { comparison_operator = "GREATER_THAN" threshold = 80 threshold_type = "PERCENTAGE" notification_type = "ACTUAL" subscriber_email_addresses = ["team@example.com"] } } ``` ### Cost Anomaly Detection - AWS Cost Anomaly Detection - Azure Cost Management alerts - GCP Budget alerts ## Architecture Patterns ### Pattern 1: Serverless First - Use Lambda/Functions for event-driven - Pay only for execution time - Auto-scaling included - No idle costs ### Pattern 2: Right-Sized Databases ``` Development: t3.small RDS Staging: t3.large RDS Production: r6g.2xlarge RDS with read replicas ``` ### Pattern 3: Multi-Tier Storage ``` Hot data: S3 Standard Warm data: S3 Standard-IA (30 days) Cold data: S3 Glacier (90 days) Archive: S3 Deep Archive (365 days) ``` ### Pattern 4: Auto-Scaling ```hcl resource "aws_autoscaling_policy" "scale_up" { name = "scale-up" scaling_adjustment = 2 adjustment_type = "ChangeInCapacity" cooldown = 300 autoscaling_group_name = aws_autoscaling_group.main.name } resource "aws_cloudwatch_metric_alarm" "cpu_high" { alarm_name = "cpu-high" comparison_operator = "GreaterThanThreshold" evaluation_periods = "2" metric_name = "CPUUtilization" namespace = "AWS/EC2" period = "60" statistic = "Average" threshold = "80" alarm_actions = [aws_autoscaling_policy.scale_up.arn] } ``` ## Cost Optimization Checklist - [ ] Implement cost allocation tags - [ ] Delete unused resources (EBS, EIPs, snapshots) - [ ] Right-size instances based on utilization - [ ] Use reserved capacity for steady workloads - [ ] Implement auto-scaling - [ ] Optimize storage classes - [ ] Use lifecycle policies - [ ] Enable cost anomaly detection - [ ] Set budget alerts - [ ] Review costs weekly - [ ] Use spot/preemptible instances - [ ] Optimize data transfer costs - [ ] Implement caching layers - [ ] Use managed services - [ ] Monitor and optimize continuously ## Tools - **AWS:** Cost Explorer, Cost Anomaly Detection, Compute Optimizer - **Azure:** Cost Management, Advisor - **GCP:** Cost Management, Recommender - **Multi-cloud:** CloudHealth, Cloudability, Kubecost ## Reference Files - `references/tagging-standards.md` - Tagging conventions - `assets/cost-analysis-template.xlsx` - Cost analysis spreadsheet ## Related Skills - `terraform-module-library` - For resource provisioning - `multi-cloud-architecture` - For cloud selection
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

istio-traffic-management

Configure Istio traffic management including routing, load

architecture
⭐1
# Istio Traffic Management Comprehensive guide to Istio traffic management for production service mesh deployments. ## When to Use This Skill - Configuring service-to-service routing - Implementing canary or blue-green deployments - Setting up circuit breakers and retries - Load balancing configuration - Traffic mirroring for testing - Fault injection for chaos engineering ## Core Concepts ### 1. Traffic Management Resources | Resource | Purpose | Scope | | ------------------- | ----------------------------- | ------------- | | **VirtualService** | Route traffic to destinations | Host-based | | **DestinationRule** | Define policies after routing | Service-based | | **Gateway** | Configure ingress/egress | Cluster edge | | **ServiceEntry** | Add external services | Mesh-wide | ### 2. Traffic Flow ``` Client β†’ Gateway β†’ VirtualService β†’ DestinationRule β†’ Service (routing) (policies) (pods) ``` ## Templates ### Template 1: Basic Routing ```yaml apiVersion: networking.istio.io/v1beta1 kind: VirtualService metadata: name: reviews-route namespace: bookinfo spec: hosts: - reviews http: - match: - headers: end-user: exact: jason route: - destination: host: reviews subset: v2 - route: - destination: host: reviews subset: v1 --- apiVersion: networking.istio.io/v1beta1 kind: DestinationRule metadata: name: reviews-destination namespace: bookinfo spec: host: reviews subsets: - name: v1 labels: version: v1 - name: v2 labels: version: v2 - name: v3 labels: version: v3 ``` ### Template 2: Canary Deployment ```yaml apiVersion: networking.istio.io/v1beta1 kind: VirtualService metadata: name: my-service-canary spec: hosts: - my-service http: - route: - destination: host: my-service subset: stable weight: 90 - destination: host: my-service subset: canary weight: 10 --- apiVersion: networking.istio.io/v1beta1 kind: DestinationRule metadata: name: my-service-dr spec: host: my-service trafficPolicy: connectionPool: tcp: maxConnections: 100 http: h2UpgradePolicy: UPGRADE http1MaxPendingRequests: 100 http2MaxRequests: 1000 subsets: - name: stable labels: version: stable - name: canary labels: version: canary ``` ### Template 3: Circuit Breaker ```yaml apiVersion: networking.istio.io/v1beta1 kind: DestinationRule metadata: name: circuit-breaker spec: host: my-service trafficPolicy: connectionPool: tcp: maxConnections: 100 http: http1MaxPendingRequests: 100 http2MaxRequests: 1000 maxRequestsPerConnection: 10 maxRetries: 3 outlierDetection: consecutive5xxErrors: 5 interval: 30s baseEjectionTime: 30s maxEjectionPercent: 50 minHealthPercent: 30 ``` ### Template 4: Retry and Timeout ```yaml apiVersion: networking.istio.io/v1beta1 kind: VirtualService metadata: name: ratings-retry spec: hosts: - ratings http: - route: - destination: host: ratings timeout: 10s retries: attempts: 3 perTryTimeout: 3s retryOn: connect-failure,refused-stream,unavailable,cancelled,retriable-4xx,503 retryRemoteLocalities: true ``` ### Template 5: Traffic Mirroring ```yaml apiVersion: networking.istio.io/v1beta1 kind: VirtualService metadata: name: mirror-traffic spec: hosts: - my-service http: - route: - destination: host: my-service subset: v1 mirror: host: my-service subset: v2 mirrorPercentage: value: 100.0 ``` ### Template 6: Fault Injection ```yaml apiVersion: networking.istio.io/v1beta1 kind: VirtualService metadata: name: fault-injection spec: hosts: - ratings http: - fault: delay: percentage: value: 10 fixedDelay: 5s abort: percentage: value: 5 httpStatus: 503 route: - destination: host: ratings ``` ### Template 7: Ingress Gateway ```yaml apiVersion: networking.istio.io/v1beta1 kind: Gateway metadata: name: my-gateway spec: selector: istio: ingressgateway servers: - port: number: 443 name: https protocol: HTTPS tls: mode: SIMPLE credentialName: my-tls-secret hosts: - "*.example.com" --- apiVersion: networking.istio.io/v1beta1 kind: VirtualService metadata: name: my-vs spec: hosts: - "api.example.com" gateways: - my-gateway http: - match: - uri: prefix: /api/v1 route: - destination: host: api-service port: number: 8080 ``` ## Load Balancing Strategies ```yaml apiVersion: networking.istio.io/v1beta1 kind: DestinationRule metadata: name: load-balancing spec: host: my-service trafficPolicy: loadBalancer: simple: ROUND_ROBIN # or LEAST_CONN, RANDOM, PASSTHROUGH --- # Consistent hashing for sticky sessions apiVersion: networking.istio.io/v1beta1 kind: DestinationRule metadata: name: sticky-sessions spec: host: my-service trafficPolicy: loadBalancer: consistentHash: httpHeaderName: x-user-id # or: httpCookie, useSourceIp, httpQueryParameterName ``` ## Best Practices ### Do's - **Start simple** - Add complexity incrementally - **Use subsets** - Version your services clearly - **Set timeouts** - Always configure reasonable timeouts - **Enable retries** - But with backoff and limits - **Monitor** - Use Kiali and Jaeger for visibility ### Don'ts - **Don't over-retry** - Can cause cascading failures - **Don't ignore outlier detection** - Enable circuit breakers - **Don't mirror to production** - Mirror to test environments - **Don't skip canary** - Test with small traffic percentage first ## Debugging Commands ```bash # Check VirtualService configuration istioctl analyze # View effective routes istioctl proxy-config routes deploy/my-app -o json # Check endpoint discovery istioctl proxy-config endpoints deploy/my-app # Debug traffic istioctl proxy-config log deploy/my-app --level debug ``` ## Resources - [Istio Traffic Management](https://istio.io/latest/docs/concepts/traffic-management/) - [Virtual Service Reference](https://istio.io/latest/docs/reference/config/networking/virtual-service/) - [Destination Rule Reference](https://istio.io/latest/docs/reference/config/networking/destination-rule/)
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

linkerd-patterns

Implement Linkerd service mesh patterns for lightweight,

architecture
⭐1
# Linkerd Patterns Production patterns for Linkerd service mesh - the lightweight, security-first service mesh for Kubernetes. ## When to Use This Skill - Setting up a lightweight service mesh - Implementing automatic mTLS - Configuring traffic splits for canary deployments - Setting up service profiles for per-route metrics - Implementing retries and timeouts - Multi-cluster service mesh ## Core Concepts ### 1. Linkerd Architecture ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Control Plane β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ destiny β”‚ β”‚ identity β”‚ β”‚ proxy-inject β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Data Plane β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚proxy│────│proxy│────│proxyβ”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”Œβ”€β”€β”΄β”€β”€β” β”Œβ”€β”€β”΄β”€β”€β” β”Œβ”€β”€β”΄β”€β”€β” β”‚ β”‚ β”‚ app β”‚ β”‚ app β”‚ β”‚ app β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”˜ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` ### 2. Key Resources | Resource | Purpose | | ----------------------- | ------------------------------------ | | **ServiceProfile** | Per-route metrics, retries, timeouts | | **TrafficSplit** | Canary deployments, A/B testing | | **Server** | Define server-side policies | | **ServerAuthorization** | Access control policies | ## Templates ### Template 1: Mesh Installation ```bash # Install CLI curl --proto '=https' --tlsv1.2 -sSfL https://run.linkerd.io/install | sh # Validate cluster linkerd check --pre # Install CRDs linkerd install --crds | kubectl apply -f - # Install control plane linkerd install | kubectl apply -f - # Verify installation linkerd check # Install viz extension (optional) linkerd viz install | kubectl apply -f - ``` ### Template 2: Inject Namespace ```yaml # Automatic injection for namespace apiVersion: v1 kind: Namespace metadata: name: my-app annotations: linkerd.io/inject: enabled --- # Or inject specific deployment apiVersion: apps/v1 kind: Deployment metadata: name: my-app annotations: linkerd.io/inject: enabled spec: template: metadata: annotations: linkerd.io/inject: enabled ``` ### Template 3: Service Profile with Retries ```yaml apiVersion: linkerd.io/v1alpha2 kind: ServiceProfile metadata: name: my-service.my-namespace.svc.cluster.local namespace: my-namespace spec: routes: - name: GET /api/users condition: method: GET pathRegex: /api/users responseClasses: - condition: status: min: 500 max: 599 isFailure: true isRetryable: true - name: POST /api/users condition: method: POST pathRegex: /api/users # POST not retryable by default isRetryable: false - name: GET /api/users/{id} condition: method: GET pathRegex: /api/users/[^/]+ timeout: 5s isRetryable: true retryBudget: retryRatio: 0.2 minRetriesPerSecond: 10 ttl: 10s ``` ### Template 4: Traffic Split (Canary) ```yaml apiVersion: split.smi-spec.io/v1alpha1 kind: TrafficSplit metadata: name: my-service-canary namespace: my-namespace spec: service: my-service backends: - service: my-service-stable weight: 900m # 90% - service: my-service-canary weight: 100m # 10% ``` ### Template 5: Server Authorization Policy ```yaml # Define the server apiVersion: policy.linkerd.io/v1beta1 kind: Server metadata: name: my-service-http namespace: my-namespace spec: podSelector: matchLabels: app: my-service port: http proxyProtocol: HTTP/1 --- # Allow traffic from specific clients apiVersion: policy.linkerd.io/v1beta1 kind: ServerAuthorization metadata: name: allow-frontend namespace: my-namespace spec: server: name: my-service-http client: meshTLS: serviceAccounts: - name: frontend namespace: my-namespace --- # Allow unauthenticated traffic (e.g., from ingress) apiVersion: policy.linkerd.io/v1beta1 kind: ServerAuthorization metadata: name: allow-ingress namespace: my-namespace spec: server: name: my-service-http client: unauthenticated: true networks: - cidr: 10.0.0.0/8 ``` ### Template 6: HTTPRoute for Advanced Routing ```yaml apiVersion: policy.linkerd.io/v1beta2 kind: HTTPRoute metadata: name: my-route namespace: my-namespace spec: parentRefs: - name: my-service kind: Service group: core port: 8080 rules: - matches: - path: type: PathPrefix value: /api/v2 - headers: - name: x-api-version value: v2 backendRefs: - name: my-service-v2 port: 8080 - matches: - path: type: PathPrefix value: /api backendRefs: - name: my-service-v1 port: 8080 ``` ### Template 7: Multi-cluster Setup ```bash # On each cluster, install with cluster credentials linkerd multicluster install | kubectl apply -f - # Link clusters linkerd multicluster link --cluster-name west \ --api-server-address https://west.example.com:6443 \ | kubectl apply -f - # Export a service to other clusters kubectl label svc/my-service mirror.linkerd.io/exported=true # Verify cross-cluster connectivity linkerd multicluster check linkerd multicluster gateways ``` ## Monitoring Commands ```bash # Live traffic view linkerd viz top deploy/my-app # Per-route metrics linkerd viz routes deploy/my-app # Check proxy status linkerd viz stat deploy -n my-namespace # View service dependencies linkerd viz edges deploy -n my-namespace # Dashboard linkerd viz dashboard ``` ## Debugging ```bash # Check injection status linkerd check --proxy -n my-namespace # View proxy logs kubectl logs deploy/my-app -c linkerd-proxy # Debug identity/TLS linkerd identity -n my-namespace # Tap traffic (live) linkerd viz tap deploy/my-app --to deploy/my-backend ``` ## Best Practices ### Do's - **Enable mTLS everywhere** - It's automatic with Linkerd - **Use ServiceProfiles** - Get per-route metrics and retries - **Set retry budgets** - Prevent retry storms - **Monitor golden metrics** - Success rate, latency, throughput ### Don'ts - **Don't skip check** - Always run `linkerd check` after changes - **Don't over-configure** - Linkerd defaults are sensible - **Don't ignore ServiceProfiles** - They unlock advanced features - **Don't forget timeouts** - Set appropriate values per route ## Resources - [Linkerd Documentation](https://linkerd.io/2.14/overview/) - [Service Profiles](https://linkerd.io/2.14/features/service-profiles/) - [Authorization Policy](https://linkerd.io/2.14/features/server-policy/)
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

mtls-configuration

Configure mutual TLS (mTLS) for zero-trust service-to-service

architecture
⭐1
# mTLS Configuration Comprehensive guide to implementing mutual TLS for zero-trust service mesh communication. ## When to Use This Skill - Implementing zero-trust networking - Securing service-to-service communication - Certificate rotation and management - Debugging TLS handshake issues - Compliance requirements (PCI-DSS, HIPAA) - Multi-cluster secure communication ## Core Concepts ### 1. mTLS Flow ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Service β”‚ β”‚ Service β”‚ β”‚ A β”‚ β”‚ B β”‚ β””β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”˜ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”΄β”€β”€β”€β”€β” TLS Handshake β”Œβ”€β”€β”€β”€β”΄β”€β”€β”€β”€β” β”‚ Proxy │◄───────────────────────────►│ Proxy β”‚ β”‚(Sidecar)β”‚ 1. ClientHello β”‚(Sidecar)β”‚ β”‚ β”‚ 2. ServerHello + Cert β”‚ β”‚ β”‚ β”‚ 3. Client Cert β”‚ β”‚ β”‚ β”‚ 4. Verify Both Certs β”‚ β”‚ β”‚ β”‚ 5. Encrypted Channel β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` ### 2. Certificate Hierarchy ``` Root CA (Self-signed, long-lived) β”‚ β”œβ”€β”€ Intermediate CA (Cluster-level) β”‚ β”‚ β”‚ β”œβ”€β”€ Workload Cert (Service A) β”‚ └── Workload Cert (Service B) β”‚ └── Intermediate CA (Multi-cluster) β”‚ └── Cross-cluster certs ``` ## Templates ### Template 1: Istio mTLS (Strict Mode) ```yaml # Enable strict mTLS mesh-wide apiVersion: security.istio.io/v1beta1 kind: PeerAuthentication metadata: name: default namespace: istio-system spec: mtls: mode: STRICT --- # Namespace-level override (permissive for migration) apiVersion: security.istio.io/v1beta1 kind: PeerAuthentication metadata: name: default namespace: legacy-namespace spec: mtls: mode: PERMISSIVE --- # Workload-specific policy apiVersion: security.istio.io/v1beta1 kind: PeerAuthentication metadata: name: payment-service namespace: production spec: selector: matchLabels: app: payment-service mtls: mode: STRICT portLevelMtls: 8080: mode: STRICT 9090: mode: DISABLE # Metrics port, no mTLS ``` ### Template 2: Istio Destination Rule for mTLS ```yaml apiVersion: networking.istio.io/v1beta1 kind: DestinationRule metadata: name: default namespace: istio-system spec: host: "*.local" trafficPolicy: tls: mode: ISTIO_MUTUAL --- # TLS to external service apiVersion: networking.istio.io/v1beta1 kind: DestinationRule metadata: name: external-api spec: host: api.external.com trafficPolicy: tls: mode: SIMPLE caCertificates: /etc/certs/external-ca.pem --- # Mutual TLS to external service apiVersion: networking.istio.io/v1beta1 kind: DestinationRule metadata: name: partner-api spec: host: api.partner.com trafficPolicy: tls: mode: MUTUAL clientCertificate: /etc/certs/client.pem privateKey: /etc/certs/client-key.pem caCertificates: /etc/certs/partner-ca.pem ``` ### Template 3: Cert-Manager with Istio ```yaml # Install cert-manager issuer for Istio apiVersion: cert-manager.io/v1 kind: ClusterIssuer metadata: name: istio-ca spec: ca: secretName: istio-ca-secret --- # Create Istio CA secret apiVersion: v1 kind: Secret metadata: name: istio-ca-secret namespace: cert-manager type: kubernetes.io/tls data: tls.crt: <base64-encoded-ca-cert> tls.key: <base64-encoded-ca-key> --- # Certificate for workload apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: my-service-cert namespace: my-namespace spec: secretName: my-service-tls duration: 24h renewBefore: 8h issuerRef: name: istio-ca kind: ClusterIssuer commonName: my-service.my-namespace.svc.cluster.local dnsNames: - my-service - my-service.my-namespace - my-service.my-namespace.svc - my-service.my-namespace.svc.cluster.local usages: - server auth - client auth ``` ### Template 4: SPIFFE/SPIRE Integration ```yaml # SPIRE Server configuration apiVersion: v1 kind: ConfigMap metadata: name: spire-server namespace: spire data: server.conf: | server { bind_address = "0.0.0.0" bind_port = "8081" trust_domain = "example.org" data_dir = "/run/spire/data" log_level = "INFO" ca_ttl = "168h" default_x509_svid_ttl = "1h" } plugins { DataStore "sql" { plugin_data { database_type = "sqlite3" connection_string = "/run/spire/data/datastore.sqlite3" } } NodeAttestor "k8s_psat" { plugin_data { clusters = { "demo-cluster" = { service_account_allow_list = ["spire:spire-agent"] } } } } KeyManager "memory" { plugin_data {} } UpstreamAuthority "disk" { plugin_data { key_file_path = "/run/spire/secrets/bootstrap.key" cert_file_path = "/run/spire/secrets/bootstrap.crt" } } } --- # SPIRE Agent DaemonSet (abbreviated) apiVersion: apps/v1 kind: DaemonSet metadata: name: spire-agent namespace: spire spec: selector: matchLabels: app: spire-agent template: spec: containers: - name: spire-agent image: ghcr.io/spiffe/spire-agent:1.8.0 volumeMounts: - name: spire-agent-socket mountPath: /run/spire/sockets volumes: - name: spire-agent-socket hostPath: path: /run/spire/sockets type: DirectoryOrCreate ``` ### Template 5: Linkerd mTLS (Automatic) ```yaml # Linkerd enables mTLS automatically # Verify with: # linkerd viz edges deployment -n my-namespace # For external services without mTLS apiVersion: policy.linkerd.io/v1beta1 kind: Server metadata: name: external-api namespace: my-namespace spec: podSelector: matchLabels: app: my-app port: external-api proxyProtocol: HTTP/1 # or TLS for passthrough --- # Skip TLS for specific port apiVersion: v1 kind: Service metadata: name: my-service annotations: config.linkerd.io/skip-outbound-ports: "3306" # MySQL ``` ## Certificate Rotation ```bash # Istio - Check certificate expiry istioctl proxy-config secret deploy/my-app -o json | \ jq '.dynamicActiveSecrets[0].secret.tlsCertificate.certificateChain.inlineBytes' | \ tr -d '"' | base64 -d | openssl x509 -text -noout # Force certificate rotation kubectl rollout restart deployment/my-app # Check Linkerd identity linkerd identity -n my-namespace ``` ## Debugging mTLS Issues ```bash # Istio - Check if mTLS is enabled istioctl authn tls-check my-service.my-namespace.svc.cluster.local # Verify peer authentication kubectl get peerauthentication --all-namespaces # Check destination rules kubectl get destinationrule --all-namespaces # Debug TLS handshake istioctl proxy-config log deploy/my-app --level debug kubectl logs deploy/my-app -c istio-proxy | grep -i tls # Linkerd - Check mTLS status linkerd viz edges deployment -n my-namespace linkerd viz tap deploy/my-app --to deploy/my-backend ``` ## Best Practices ### Do's - **Start with PERMISSIVE** - Migrate gradually to STRICT - **Monitor certificate expiry** - Set up alerts - **Use short-lived certs** - 24h or less for workloads - **Rotate CA periodically** - Plan for CA rotation - **Log TLS errors** - For debugging and audit ### Don'ts - **Don't disable mTLS** - For convenience in production - **Don't ignore cert expiry** - Automate rotation - **Don't use self-signed certs** - Use proper CA hierarchy - **Don't skip verification** - Verify the full chain ## Resources - [Istio Security](https://istio.io/latest/docs/concepts/security/) - [SPIFFE/SPIRE](https://spiffe.io/) - [cert-manager](https://cert-manager.io/) - [Zero Trust Architecture (NIST)](https://www.nist.gov/publications/zero-trust-architecture)
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

multi-cloud-architecture

Design multi-cloud architectures using a decision framework to

architecture
⭐1
# Multi-Cloud Architecture Decision framework and patterns for architecting applications across AWS, Azure, and GCP. ## Purpose Design cloud-agnostic architectures and make informed decisions about service selection across cloud providers. ## When to Use - Design multi-cloud strategies - Migrate between cloud providers - Select cloud services for specific workloads - Implement cloud-agnostic architectures - Optimize costs across providers ## Cloud Service Comparison ### Compute Services | AWS | Azure | GCP | Use Case | | ------- | ------------------- | --------------- | ------------------ | | EC2 | Virtual Machines | Compute Engine | IaaS VMs | | ECS | Container Instances | Cloud Run | Containers | | EKS | AKS | GKE | Kubernetes | | Lambda | Functions | Cloud Functions | Serverless | | Fargate | Container Apps | Cloud Run | Managed containers | ### Storage Services | AWS | Azure | GCP | Use Case | | ------- | --------------- | --------------- | -------------- | | S3 | Blob Storage | Cloud Storage | Object storage | | EBS | Managed Disks | Persistent Disk | Block storage | | EFS | Azure Files | Filestore | File storage | | Glacier | Archive Storage | Archive Storage | Cold storage | ### Database Services | AWS | Azure | GCP | Use Case | | ----------- | ---------------- | ------------- | --------------- | | RDS | SQL Database | Cloud SQL | Managed SQL | | DynamoDB | Cosmos DB | Firestore | NoSQL | | Aurora | PostgreSQL/MySQL | Cloud Spanner | Distributed SQL | | ElastiCache | Cache for Redis | Memorystore | Caching | **Reference:** See `references/service-comparison.md` for complete comparison ## Multi-Cloud Patterns ### Pattern 1: Single Provider with DR - Primary workload in one cloud - Disaster recovery in another - Database replication across clouds - Automated failover ### Pattern 2: Best-of-Breed - Use best service from each provider - AI/ML on GCP - Enterprise apps on Azure - General compute on AWS ### Pattern 3: Geographic Distribution - Serve users from nearest cloud region - Data sovereignty compliance - Global load balancing - Regional failover ### Pattern 4: Cloud-Agnostic Abstraction - Kubernetes for compute - PostgreSQL for database - S3-compatible storage (MinIO) - Open source tools ## Cloud-Agnostic Architecture ### Use Cloud-Native Alternatives - **Compute:** Kubernetes (EKS/AKS/GKE) - **Database:** PostgreSQL/MySQL (RDS/SQL Database/Cloud SQL) - **Message Queue:** Apache Kafka (MSK/Event Hubs/Confluent) - **Cache:** Redis (ElastiCache/Azure Cache/Memorystore) - **Object Storage:** S3-compatible API - **Monitoring:** Prometheus/Grafana - **Service Mesh:** Istio/Linkerd ### Abstraction Layers ``` Application Layer ↓ Infrastructure Abstraction (Terraform) ↓ Cloud Provider APIs ↓ AWS / Azure / GCP ``` ## Cost Comparison ### Compute Pricing Factors - **AWS:** On-demand, Reserved, Spot, Savings Plans - **Azure:** Pay-as-you-go, Reserved, Spot - **GCP:** On-demand, Committed use, Preemptible ### Cost Optimization Strategies 1. Use reserved/committed capacity (30-70% savings) 2. Leverage spot/preemptible instances 3. Right-size resources 4. Use serverless for variable workloads 5. Optimize data transfer costs 6. Implement lifecycle policies 7. Use cost allocation tags 8. Monitor with cloud cost tools **Reference:** See `references/multi-cloud-patterns.md` ## Migration Strategy ### Phase 1: Assessment - Inventory current infrastructure - Identify dependencies - Assess cloud compatibility - Estimate costs ### Phase 2: Pilot - Select pilot workload - Implement in target cloud - Test thoroughly - Document learnings ### Phase 3: Migration - Migrate workloads incrementally - Maintain dual-run period - Monitor performance - Validate functionality ### Phase 4: Optimization - Right-size resources - Implement cloud-native services - Optimize costs - Enhance security ## Best Practices 1. **Use infrastructure as code** (Terraform/OpenTofu) 2. **Implement CI/CD pipelines** for deployments 3. **Design for failure** across clouds 4. **Use managed services** when possible 5. **Implement comprehensive monitoring** 6. **Automate cost optimization** 7. **Follow security best practices** 8. **Document cloud-specific configurations** 9. **Test disaster recovery** procedures 10. **Train teams** on multiple clouds ## Reference Files - `references/service-comparison.md` - Complete service comparison - `references/multi-cloud-patterns.md` - Architecture patterns ## Related Skills - `terraform-module-library` - For IaC implementation - `cost-optimization` - For cost management - `hybrid-cloud-networking` - For connectivity
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

postgresql-table-design

Design a PostgreSQL-specific schema. Covers best-practices, data

data
⭐1
# PostgreSQL Table Design ## Core Rules - Define a **PRIMARY KEY** for reference tables (users, orders, etc.). Not always needed for time-series/event/log data. When used, prefer `BIGINT GENERATED ALWAYS AS IDENTITY`; use `UUID` only when global uniqueness/opacity is needed. - **Normalize first (to 3NF)** to eliminate data redundancy and update anomalies; denormalize **only** for measured, high-ROI reads where join performance is proven problematic. Premature denormalization creates maintenance burden. - Add **NOT NULL** everywhere it’s semantically required; use **DEFAULT**s for common values. - Create **indexes for access paths you actually query**: PK/unique (auto), **FK columns (manual!)**, frequent filters/sorts, and join keys. - Prefer **TIMESTAMPTZ** for event time; **NUMERIC** for money; **TEXT** for strings; **BIGINT** for integer values, **DOUBLE PRECISION** for floats (or `NUMERIC` for exact decimal arithmetic). ## PostgreSQL β€œGotchas” - **Identifiers**: unquoted β†’ lowercased. Avoid quoted/mixed-case names. Convention: use `snake_case` for table/column names. - **Unique + NULLs**: UNIQUE allows multiple NULLs. Use `UNIQUE (...) NULLS NOT DISTINCT` (PG15+) to restrict to one NULL. - **FK indexes**: PostgreSQL **does not** auto-index FK columns. Add them. - **No silent coercions**: length/precision overflows error out (no truncation). Example: inserting 999 into `NUMERIC(2,0)` fails with error, unlike some databases that silently truncate or round. - **Sequences/identity have gaps** (normal; don't "fix"). Rollbacks, crashes, and concurrent transactions create gaps in ID sequences (1, 2, 5, 6...). This is expected behaviorβ€”don't try to make IDs consecutive. - **Heap storage**: no clustered PK by default (unlike SQL Server/MySQL InnoDB); `CLUSTER` is one-off reorganization, not maintained on subsequent inserts. Row order on disk is insertion order unless explicitly clustered. - **MVCC**: updates/deletes leave dead tuples; vacuum handles themβ€”design to avoid hot wide-row churn. ## Data Types - **IDs**: `BIGINT GENERATED ALWAYS AS IDENTITY` preferred (`GENERATED BY DEFAULT` also fine); `UUID` when merging/federating/used in a distributed system or for opaque IDs. Generate with `uuidv7()` (preferred if using PG18+) or `gen_random_uuid()` (if using an older PG version). - **Integers**: prefer `BIGINT` unless storage space is critical; `INTEGER` for smaller ranges; avoid `SMALLINT` unless constrained. - **Floats**: prefer `DOUBLE PRECISION` over `REAL` unless storage space is critical. Use `NUMERIC` for exact decimal arithmetic. - **Strings**: prefer `TEXT`; if length limits needed, use `CHECK (LENGTH(col) <= n)` instead of `VARCHAR(n)`; avoid `CHAR(n)`. Use `BYTEA` for binary data. Large strings/binary (>2KB default threshold) automatically stored in TOAST with compression. TOAST storage: `PLAIN` (no TOAST), `EXTENDED` (compress + out-of-line), `EXTERNAL` (out-of-line, no compress), `MAIN` (compress, keep in-line if possible). Default `EXTENDED` usually optimal. Control with `ALTER TABLE tbl ALTER COLUMN col SET STORAGE strategy` and `ALTER TABLE tbl SET (toast_tuple_target = 4096)` for threshold. Case-insensitive: for locale/accent handling use non-deterministic collations; for plain ASCII use expression indexes on `LOWER(col)` (preferred unless column needs case-insensitive PK/FK/UNIQUE) or `CITEXT`. - **Money**: `NUMERIC(p,s)` (never float). - **Time**: `TIMESTAMPTZ` for timestamps; `DATE` for date-only; `INTERVAL` for durations. Avoid `TIMESTAMP` (without timezone). Use `now()` for transaction start time, `clock_timestamp()` for current wall-clock time. - **Booleans**: `BOOLEAN` with `NOT NULL` constraint unless tri-state values are required. - **Enums**: `CREATE TYPE ... AS ENUM` for small, stable sets (e.g. US states, days of week). For business-logic-driven and evolving values (e.g. order statuses) β†’ use TEXT (or INT) + CHECK or lookup table. - **Arrays**: `TEXT[]`, `INTEGER[]`, etc. Use for ordered lists where you query elements. Index with **GIN** for containment (`@>`, `<@`) and overlap (`&&`) queries. Access: `arr[1]` (1-indexed), `arr[1:3]` (slicing). Good for tags, categories; avoid for relationsβ€”use junction tables instead. Literal syntax: `'{val1,val2}'` or `ARRAY[val1,val2]`. - **Range types**: `daterange`, `numrange`, `tstzrange` for intervals. Support overlap (`&&`), containment (`@>`), operators. Index with **GiST**. Good for scheduling, versioning, numeric ranges. Pick a bounds scheme and use it consistently; prefer `[)` (inclusive/exclusive) by default. - **Network types**: `INET` for IP addresses, `CIDR` for network ranges, `MACADDR` for MAC addresses. Support network operators (`<<`, `>>`, `&&`). - **Geometric types**: `POINT`, `LINE`, `POLYGON`, `CIRCLE` for 2D spatial data. Index with **GiST**. Consider **PostGIS** for advanced spatial features. - **Text search**: `TSVECTOR` for full-text search documents, `TSQUERY` for search queries. Index `tsvector` with **GIN**. Always specify language: `to_tsvector('english', col)` and `to_tsquery('english', 'query')`. Never use single-argument versions. This applies to both index expressions and queries. - **Domain types**: `CREATE DOMAIN email AS TEXT CHECK (VALUE ~ '^[^@]+@[^@]+$')` for reusable custom types with validation. Enforces constraints across tables. - **Composite types**: `CREATE TYPE address AS (street TEXT, city TEXT, zip TEXT)` for structured data within columns. Access with `(col).field` syntax. - **JSONB**: preferred over JSON; index with **GIN**. Use only for optional/semi-structured attrs. ONLY use JSON if the original ordering of the contents MUST be preserved. - **Vector types**: `vector` type by `pgvector` for vector similarity search for embeddings. ### Do not use the following data types - DO NOT use `timestamp` (without time zone); DO use `timestamptz` instead. - DO NOT use `char(n)` or `varchar(n)`; DO use `text` instead. - DO NOT use `money` type; DO use `numeric` instead. - DO NOT use `timetz` type; DO use `timestamptz` instead. - DO NOT use `timestamptz(0)` or any other precision specification; DO use `timestamptz` instead - DO NOT use `serial` type; DO use `generated always as identity` instead. ## Table Types - **Regular**: default; fully durable, logged. - **TEMPORARY**: session-scoped, auto-dropped, not logged. Faster for scratch work. - **UNLOGGED**: persistent but not crash-safe. Faster writes; good for caches/staging. ## Row-Level Security Enable with `ALTER TABLE tbl ENABLE ROW LEVEL SECURITY`. Create policies: `CREATE POLICY user_access ON orders FOR SELECT TO app_users USING (user_id = current_user_id())`. Built-in user-based access control at the row level. ## Constraints - **PK**: implicit UNIQUE + NOT NULL; creates a B-tree index. - **FK**: specify `ON DELETE/UPDATE` action (`CASCADE`, `RESTRICT`, `SET NULL`, `SET DEFAULT`). Add explicit index on referencing columnβ€”speeds up joins and prevents locking issues on parent deletes/updates. Use `DEFERRABLE INITIALLY DEFERRED` for circular FK dependencies checked at transaction end. - **UNIQUE**: creates a B-tree index; allows multiple NULLs unless `NULLS NOT DISTINCT` (PG15+). Standard behavior: `(1, NULL)` and `(1, NULL)` are allowed. With `NULLS NOT DISTINCT`: only one `(1, NULL)` allowed. Prefer `NULLS NOT DISTINCT` unless you specifically need duplicate NULLs. - **CHECK**: row-local constraints; NULL values pass the check (three-valued logic). Example: `CHECK (price > 0)` allows NULL prices. Combine with `NOT NULL` to enforce: `price NUMERIC NOT NULL CHECK (price > 0)`. - **EXCLUDE**: prevents overlapping values using operators. `EXCLUDE USING gist (room_id WITH =, booking_period WITH &&)` prevents double-booking rooms. Requires appropriate index type (often GiST). ## Indexing - **B-tree**: default for equality/range queries (`=`, `<`, `>`, `BETWEEN`, `ORDER BY`) - **Composite**: order mattersβ€”index used if equality on leftmost prefix (`WHERE a = ? AND b > ?` uses index on `(a,b)`, but `WHERE b = ?` does not). Put most selective/frequently filtered columns first. - **Covering**: `CREATE INDEX ON tbl (id) INCLUDE (name, email)` - includes non-key columns for index-only scans without visiting table. - **Partial**: for hot subsets (`WHERE status = 'active'` β†’ `CREATE INDEX ON tbl (user_id) WHERE status = 'active'`). Any query with `status = 'active'` can use this index. - **Expression**: for computed search keys (`CREATE INDEX ON tbl (LOWER(email))`). Expression must match exactly in WHERE clause: `WHERE LOWER(email) = 'user@example.com'`. - **GIN**: JSONB containment/existence, arrays (`@>`, `?`), full-text search (`@@`) - **GiST**: ranges, geometry, exclusion constraints - **BRIN**: very large, naturally ordered data (time-series)β€”minimal storage overhead. Effective when row order on disk correlates with indexed column (insertion order or after `CLUSTER`). ## Partitioning - Use for very large tables (>100M rows) where queries consistently filter on partition key (often time/date). - Alternate use: use for tables where data maintenance tasks dictates e.g. data pruned or bulk replaced periodically - **RANGE**: common for time-series (`PARTITION BY RANGE (created_at)`). Create partitions: `CREATE TABLE logs_2024_01 PARTITION OF logs FOR VALUES FROM ('2024-01-01') TO ('2024-02-01')`. **TimescaleDB** automates time-based or ID-based partitioning with retention policies and compression. - **LIST**: for discrete values (`PARTITION BY LIST (region)`). Example: `FOR VALUES IN ('us-east', 'us-west')`. - **HASH**: for even distribution when no natural key (`PARTITION BY HASH (user_id)`). Creates N partitions with modulus. - **Constraint exclusion**: requires `CHECK` constraints on partitions for query planner to prune. Auto-created for declarative partitioning (PG10+). - Prefer declarative partitioning or hypertables. Do NOT use table inheritance. - **Limitations**: no global UNIQUE constraintsβ€”include partition key in PK/UNIQUE. FKs from partitioned tables not supported; use triggers. ## Special Considerations ### Update-Heavy Tables - **Separate hot/cold columns**β€”put frequently updated columns in separate table to minimize bloat. - **Use `fillfactor=90`** to leave space for HOT updates that avoid index maintenance. - **Avoid updating indexed columns**β€”prevents beneficial HOT updates. - **Partition by update patterns**β€”separate frequently updated rows in a different partition from stable data. ### Insert-Heavy Workloads - **Minimize indexes**β€”only create what you query; every index slows inserts. - **Use `COPY` or multi-row `INSERT`** instead of single-row inserts. - **UNLOGGED tables** for rebuildable staging dataβ€”much faster writes. - **Defer index creation** for bulk loadsβ€”>drop index, load data, recreate indexes. - **Partition by time/hash** to distribute load. **TimescaleDB** automates partitioning and compression of insert-heavy data. - **Use a natural key for primary key** such as a (timestamp, device_id) if enforcing global uniqueness is important many insert-heavy tables don't need a primary key at all. - If you do need a surrogate key, **Prefer `BIGINT GENERATED ALWAYS AS IDENTITY` over `UUID`**. ### Upsert-Friendly Design - **Requires UNIQUE index** on conflict target columnsβ€”`ON CONFLICT (col1, col2)` needs exact matching unique index (partial indexes don't work). - **Use `EXCLUDED.column`** to reference would-be-inserted values; only update columns that actually changed to reduce write overhead. - **`DO NOTHING` faster** than `DO UPDATE` when no actual update needed. ### Safe Schema Evolution - **Transactional DDL**: most DDL operations can run in transactions and be rolled backβ€”`BEGIN; ALTER TABLE...; ROLLBACK;` for safe testing. - **Concurrent index creation**: `CREATE INDEX CONCURRENTLY` avoids blocking writes but can't run in transactions. - **Volatile defaults cause rewrites**: adding `NOT NULL` columns with volatile defaults (e.g., `now()`, `gen_random_uuid()`) rewrites entire table. Non-volatile defaults are fast. - **Drop constraints before columns**: `ALTER TABLE DROP CONSTRAINT` then `DROP COLUMN` to avoid dependency issues. - **Function signature changes**: `CREATE OR REPLACE` with different arguments creates overloads, not replacements. DROP old version if no overload desired. ## Generated Columns - `... GENERATED ALWAYS AS (<expr>) STORED` for computed, indexable fields. PG18+ adds `VIRTUAL` columns (computed on read, not stored). ## Extensions - **`pgcrypto`**: `crypt()` for password hashing. - **`uuid-ossp`**: alternative UUID functions; prefer `pgcrypto` for new projects. - **`pg_trgm`**: fuzzy text search with `%` operator, `similarity()` function. Index with GIN for `LIKE '%pattern%'` acceleration. - **`citext`**: case-insensitive text type. Prefer expression indexes on `LOWER(col)` unless you need case-insensitive constraints. - **`btree_gin`/`btree_gist`**: enable mixed-type indexes (e.g., GIN index on both JSONB and text columns). - **`hstore`**: key-value pairs; mostly superseded by JSONB but useful for simple string mappings. - **`timescaledb`**: essential for time-seriesβ€”automated partitioning, retention, compression, continuous aggregates. - **`postgis`**: comprehensive geospatial support beyond basic geometric typesβ€”essential for location-based applications. - **`pgvector`**: vector similarity search for embeddings. - **`pgaudit`**: audit logging for all database activity. ## JSONB Guidance - Prefer `JSONB` with **GIN** index. - Default: `CREATE INDEX ON tbl USING GIN (jsonb_col);` β†’ accelerates: - **Containment** `jsonb_col @> '{"k":"v"}'` - **Key existence** `jsonb_col ? 'k'`, **any/all keys** `?\|`, `?&` - **Path containment** on nested docs - **Disjunction** `jsonb_col @> ANY(ARRAY['{"status":"active"}', '{"status":"pending"}'])` - Heavy `@>` workloads: consider opclass `jsonb_path_ops` for smaller/faster containment-only indexes: - `CREATE INDEX ON tbl USING GIN (jsonb_col jsonb_path_ops);` - **Trade-off**: loses support for key existence (`?`, `?|`, `?&`) queriesβ€”only supports containment (`@>`) - Equality/range on a specific scalar field: extract and index with B-tree (generated column or expression): - `ALTER TABLE tbl ADD COLUMN price INT GENERATED ALWAYS AS ((jsonb_col->>'price')::INT) STORED;` - `CREATE INDEX ON tbl (price);` - Prefer queries like `WHERE price BETWEEN 100 AND 500` (uses B-tree) over `WHERE (jsonb_col->>'price')::INT BETWEEN 100 AND 500` without index. - Arrays inside JSONB: use GIN + `@>` for containment (e.g., tags). Consider `jsonb_path_ops` if only doing containment. - Keep core relations in tables; use JSONB for optional/variable attributes. - Use constraints to limit allowed JSONB values in a column e.g. `config JSONB NOT NULL CHECK(jsonb_typeof(config) = 'object')` ## Examples ### Users ```sql CREATE TABLE users ( user_id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY, email TEXT NOT NULL UNIQUE, name TEXT NOT NULL, created_at TIMESTAMPTZ NOT NULL DEFAULT now() ); CREATE UNIQUE INDEX ON users (LOWER(email)); CREATE INDEX ON users (created_at); ``` ### Orders ```sql CREATE TABLE orders ( order_id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY, user_id BIGINT NOT NULL REFERENCES users(user_id), status TEXT NOT NULL DEFAULT 'PENDING' CHECK (status IN ('PENDING','PAID','CANCELED')), total NUMERIC(10,2) NOT NULL CHECK (total > 0), created_at TIMESTAMPTZ NOT NULL DEFAULT now() ); CREATE INDEX ON orders (user_id); CREATE INDEX ON orders (created_at); ``` ### JSONB ```sql CREATE TABLE profiles ( user_id BIGINT PRIMARY KEY REFERENCES users(user_id), attrs JSONB NOT NULL DEFAULT '{}', theme TEXT GENERATED ALWAYS AS (attrs->>'theme') STORED ); CREATE INDEX profiles_attrs_gin ON profiles USING GIN (attrs); ```
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

auth-implementation-patterns

Master authentication and authorization patterns including JWT,

coding
⭐1
# Authentication & Authorization Implementation Patterns Build secure, scalable authentication and authorization systems using industry-standard patterns and modern best practices. ## When to Use This Skill - Implementing user authentication systems - Securing REST or GraphQL APIs - Adding OAuth2/social login - Implementing role-based access control (RBAC) - Designing session management - Migrating authentication systems - Debugging auth issues - Implementing SSO or multi-tenancy ## Core Concepts ### 1. Authentication vs Authorization **Authentication (AuthN)**: Who are you? - Verifying identity (username/password, OAuth, biometrics) - Issuing credentials (sessions, tokens) - Managing login/logout **Authorization (AuthZ)**: What can you do? - Permission checking - Role-based access control (RBAC) - Resource ownership validation - Policy enforcement ### 2. Authentication Strategies **Session-Based:** - Server stores session state - Session ID in cookie - Traditional, simple, stateful **Token-Based (JWT):** - Stateless, self-contained - Scales horizontally - Can store claims **OAuth2/OpenID Connect:** - Delegate authentication - Social login (Google, GitHub) - Enterprise SSO ## JWT Authentication ### Pattern 1: JWT Implementation ```typescript // JWT structure: header.payload.signature import jwt from "jsonwebtoken"; import { Request, Response, NextFunction } from "express"; interface JWTPayload { userId: string; email: string; role: string; iat: number; exp: number; } // Generate JWT function generateTokens(userId: string, email: string, role: string) { const accessToken = jwt.sign( { userId, email, role }, process.env.JWT_SECRET!, { expiresIn: "15m" }, // Short-lived ); const refreshToken = jwt.sign( { userId }, process.env.JWT_REFRESH_SECRET!, { expiresIn: "7d" }, // Long-lived ); return { accessToken, refreshToken }; } // Verify JWT function verifyToken(token: string): JWTPayload { try { return jwt.verify(token, process.env.JWT_SECRET!) as JWTPayload; } catch (error) { if (error instanceof jwt.TokenExpiredError) { throw new Error("Token expired"); } if (error instanceof jwt.JsonWebTokenError) { throw new Error("Invalid token"); } throw error; } } // Middleware function authenticate(req: Request, res: Response, next: NextFunction) { const authHeader = req.headers.authorization; if (!authHeader?.startsWith("Bearer ")) { return res.status(401).json({ error: "No token provided" }); } const token = authHeader.substring(7); try { const payload = verifyToken(token); req.user = payload; // Attach user to request next(); } catch (error) { return res.status(401).json({ error: "Invalid token" }); } } // Usage app.get("/api/profile", authenticate, (req, res) => { res.json({ user: req.user }); }); ``` ### Pattern 2: Refresh Token Flow ```typescript interface StoredRefreshToken { token: string; userId: string; expiresAt: Date; createdAt: Date; } class RefreshTokenService { // Store refresh token in database async storeRefreshToken(userId: string, refreshToken: string) { const expiresAt = new Date(Date.now() + 7 * 24 * 60 * 60 * 1000); await db.refreshTokens.create({ token: await hash(refreshToken), // Hash before storing userId, expiresAt, }); } // Refresh access token async refreshAccessToken(refreshToken: string) { // Verify refresh token let payload; try { payload = jwt.verify(refreshToken, process.env.JWT_REFRESH_SECRET!) as { userId: string; }; } catch { throw new Error("Invalid refresh token"); } // Check if token exists in database const storedToken = await db.refreshTokens.findOne({ where: { token: await hash(refreshToken), userId: payload.userId, expiresAt: { $gt: new Date() }, }, }); if (!storedToken) { throw new Error("Refresh token not found or expired"); } // Get user const user = await db.users.findById(payload.userId); if (!user) { throw new Error("User not found"); } // Generate new access token const accessToken = jwt.sign( { userId: user.id, email: user.email, role: user.role }, process.env.JWT_SECRET!, { expiresIn: "15m" }, ); return { accessToken }; } // Revoke refresh token (logout) async revokeRefreshToken(refreshToken: string) { await db.refreshTokens.deleteOne({ token: await hash(refreshToken), }); } // Revoke all user tokens (logout all devices) async revokeAllUserTokens(userId: string) { await db.refreshTokens.deleteMany({ userId }); } } // API endpoints app.post("/api/auth/refresh", async (req, res) => { const { refreshToken } = req.body; try { const { accessToken } = await refreshTokenService.refreshAccessToken(refreshToken); res.json({ accessToken }); } catch (error) { res.status(401).json({ error: "Invalid refresh token" }); } }); app.post("/api/auth/logout", authenticate, async (req, res) => { const { refreshToken } = req.body; await refreshTokenService.revokeRefreshToken(refreshToken); res.json({ message: "Logged out successfully" }); }); ``` ## Session-Based Authentication ### Pattern 1: Express Session ```typescript import session from "express-session"; import RedisStore from "connect-redis"; import { createClient } from "redis"; // Setup Redis for session storage const redisClient = createClient({ url: process.env.REDIS_URL, }); await redisClient.connect(); app.use( session({ store: new RedisStore({ client: redisClient }), secret: process.env.SESSION_SECRET!, resave: false, saveUninitialized: false, cookie: { secure: process.env.NODE_ENV === "production", // HTTPS only httpOnly: true, // No JavaScript access maxAge: 24 * 60 * 60 * 1000, // 24 hours sameSite: "strict", // CSRF protection }, }), ); // Login app.post("/api/auth/login", async (req, res) => { const { email, password } = req.body; const user = await db.users.findOne({ email }); if (!user || !(await verifyPassword(password, user.passwordHash))) { return res.status(401).json({ error: "Invalid credentials" }); } // Store user in session req.session.userId = user.id; req.session.role = user.role; res.json({ user: { id: user.id, email: user.email, role: user.role } }); }); // Session middleware function requireAuth(req: Request, res: Response, next: NextFunction) { if (!req.session.userId) { return res.status(401).json({ error: "Not authenticated" }); } next(); } // Protected route app.get("/api/profile", requireAuth, async (req, res) => { const user = await db.users.findById(req.session.userId); res.json({ user }); }); // Logout app.post("/api/auth/logout", (req, res) => { req.session.destroy((err) => { if (err) { return res.status(500).json({ error: "Logout failed" }); } res.clearCookie("connect.sid"); res.json({ message: "Logged out successfully" }); }); }); ``` ## OAuth2 / Social Login ### Pattern 1: OAuth2 with Passport.js ```typescript import passport from "passport"; import { Strategy as GoogleStrategy } from "passport-google-oauth20"; import { Strategy as GitHubStrategy } from "passport-github2"; // Google OAuth passport.use( new GoogleStrategy( { clientID: process.env.GOOGLE_CLIENT_ID!, clientSecret: process.env.GOOGLE_CLIENT_SECRET!, callbackURL: "/api/auth/google/callback", }, async (accessToken, refreshToken, profile, done) => { try { // Find or create user let user = await db.users.findOne({ googleId: profile.id, }); if (!user) { user = await db.users.create({ googleId: profile.id, email: profile.emails?.[0]?.value, name: profile.displayName, avatar: profile.photos?.[0]?.value, }); } return done(null, user); } catch (error) { return done(error, undefined); } }, ), ); // Routes app.get( "/api/auth/google", passport.authenticate("google", { scope: ["profile", "email"], }), ); app.get( "/api/auth/google/callback", passport.authenticate("google", { session: false }), (req, res) => { // Generate JWT const tokens = generateTokens(req.user.id, req.user.email, req.user.role); // Redirect to frontend with token res.redirect( `${process.env.FRONTEND_URL}/auth/callback?token=${tokens.accessToken}`, ); }, ); ``` ## Authorization Patterns ### Pattern 1: Role-Based Access Control (RBAC) ```typescript enum Role { USER = "user", MODERATOR = "moderator", ADMIN = "admin", } const roleHierarchy: Record<Role, Role[]> = { [Role.ADMIN]: [Role.ADMIN, Role.MODERATOR, Role.USER], [Role.MODERATOR]: [Role.MODERATOR, Role.USER], [Role.USER]: [Role.USER], }; function hasRole(userRole: Role, requiredRole: Role): boolean { return roleHierarchy[userRole].includes(requiredRole); } // Middleware function requireRole(...roles: Role[]) { return (req: Request, res: Response, next: NextFunction) => { if (!req.user) { return res.status(401).json({ error: "Not authenticated" }); } if (!roles.some((role) => hasRole(req.user.role, role))) { return res.status(403).json({ error: "Insufficient permissions" }); } next(); }; } // Usage app.delete( "/api/users/:id", authenticate, requireRole(Role.ADMIN), async (req, res) => { // Only admins can delete users await db.users.delete(req.params.id); res.json({ message: "User deleted" }); }, ); ``` ### Pattern 2: Permission-Based Access Control ```typescript enum Permission { READ_USERS = "read:users", WRITE_USERS = "write:users", DELETE_USERS = "delete:users", READ_POSTS = "read:posts", WRITE_POSTS = "write:posts", } const rolePermissions: Record<Role, Permission[]> = { [Role.USER]: [Permission.READ_POSTS, Permission.WRITE_POSTS], [Role.MODERATOR]: [ Permission.READ_POSTS, Permission.WRITE_POSTS, Permission.READ_USERS, ], [Role.ADMIN]: Object.values(Permission), }; function hasPermission(userRole: Role, permission: Permission): boolean { return rolePermissions[userRole]?.includes(permission) ?? false; } function requirePermission(...permissions: Permission[]) { return (req: Request, res: Response, next: NextFunction) => { if (!req.user) { return res.status(401).json({ error: "Not authenticated" }); } const hasAllPermissions = permissions.every((permission) => hasPermission(req.user.role, permission), ); if (!hasAllPermissions) { return res.status(403).json({ error: "Insufficient permissions" }); } next(); }; } // Usage app.get( "/api/users", authenticate, requirePermission(Permission.READ_USERS), async (req, res) => { const users = await db.users.findAll(); res.json({ users }); }, ); ``` ### Pattern 3: Resource Ownership ```typescript // Check if user owns resource async function requireOwnership( resourceType: "post" | "comment", resourceIdParam: string = "id", ) { return async (req: Request, res: Response, next: NextFunction) => { if (!req.user) { return res.status(401).json({ error: "Not authenticated" }); } const resourceId = req.params[resourceIdParam]; // Admins can access anything if (req.user.role === Role.ADMIN) { return next(); } // Check ownership let resource; if (resourceType === "post") { resource = await db.posts.findById(resourceId); } else if (resourceType === "comment") { resource = await db.comments.findById(resourceId); } if (!resource) { return res.status(404).json({ error: "Resource not found" }); } if (resource.userId !== req.user.userId) { return res.status(403).json({ error: "Not authorized" }); } next(); }; } // Usage app.put( "/api/posts/:id", authenticate, requireOwnership("post"), async (req, res) => { // User can only update their own posts const post = await db.posts.update(req.params.id, req.body); res.json({ post }); }, ); ``` ## Security Best Practices ### Pattern 1: Password Security ```typescript import bcrypt from "bcrypt"; import { z } from "zod"; // Password validation schema const passwordSchema = z .string() .min(12, "Password must be at least 12 characters") .regex(/[A-Z]/, "Password must contain uppercase letter") .regex(/[a-z]/, "Password must contain lowercase letter") .regex(/[0-9]/, "Password must contain number") .regex(/[^A-Za-z0-9]/, "Password must contain special character"); // Hash password async function hashPassword(password: string): Promise<string> { const saltRounds = 12; // 2^12 iterations return bcrypt.hash(password, saltRounds); } // Verify password async function verifyPassword( password: string, hash: string, ): Promise<boolean> { return bcrypt.compare(password, hash); } // Registration with password validation app.post("/api/auth/register", async (req, res) => { try { const { email, password } = req.body; // Validate password passwordSchema.parse(password); // Check if user exists const existingUser = await db.users.findOne({ email }); if (existingUser) { return res.status(400).json({ error: "Email already registered" }); } // Hash password const passwordHash = await hashPassword(password); // Create user const user = await db.users.create({ email, passwordHash, }); // Generate tokens const tokens = generateTokens(user.id, user.email, user.role); res.status(201).json({ user: { id: user.id, email: user.email }, ...tokens, }); } catch (error) { if (error instanceof z.ZodError) { return res.status(400).json({ error: error.errors[0].message }); } res.status(500).json({ error: "Registration failed" }); } }); ``` ### Pattern 2: Rate Limiting ```typescript import rateLimit from "express-rate-limit"; import RedisStore from "rate-limit-redis"; // Login rate limiter const loginLimiter = rateLimit({ store: new RedisStore({ client: redisClient }), windowMs: 15 * 60 * 1000, // 15 minutes max: 5, // 5 attempts message: "Too many login attempts, please try again later", standardHeaders: true, legacyHeaders: false, }); // API rate limiter const apiLimiter = rateLimit({ windowMs: 60 * 1000, // 1 minute max: 100, // 100 requests per minute standardHeaders: true, }); // Apply to routes app.post("/api/auth/login", loginLimiter, async (req, res) => { // Login logic }); app.use("/api/", apiLimiter); ``` ## Best Practices 1. **Never Store Plain Passwords**: Always hash with bcrypt/argon2 2. **Use HTTPS**: Encrypt data in transit 3. **Short-Lived Access Tokens**: 15-30 minutes max 4. **Secure Cookies**: httpOnly, secure, sameSite flags 5. **Validate All Input**: Email format, password strength 6. **Rate Limit Auth Endpoints**: Prevent brute force attacks 7. **Implement CSRF Protection**: For session-based auth 8. **Rotate Secrets Regularly**: JWT secrets, session secrets 9. **Log Security Events**: Login attempts, failed auth 10. **Use MFA When Possible**: Extra security layer ## Common Pitfalls - **Weak Passwords**: Enforce strong password policies - **JWT in localStorage**: Vulnerable to XSS, use httpOnly cookies - **No Token Expiration**: Tokens should expire - **Client-Side Auth Checks Only**: Always validate server-side - **Insecure Password Reset**: Use secure tokens with expiration - **No Rate Limiting**: Vulnerable to brute force - **Trusting Client Data**: Always validate on server ## Resources - **references/jwt-best-practices.md**: JWT implementation guide - **references/oauth2-flows.md**: OAuth2 flow diagrams and examples - **references/session-security.md**: Secure session management - **assets/auth-security-checklist.md**: Security review checklist - **assets/password-policy-template.md**: Password requirements template - **scripts/token-validator.ts**: JWT validation utility
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

architecture-decision-records

Write and maintain Architecture Decision Records (ADRs) following

coding
⭐1
# Architecture Decision Records Comprehensive patterns for creating, maintaining, and managing Architecture Decision Records (ADRs) that capture the context and rationale behind significant technical decisions. ## When to Use This Skill - Making significant architectural decisions - Documenting technology choices - Recording design trade-offs - Onboarding new team members - Reviewing historical decisions - Establishing decision-making processes ## Core Concepts ### 1. What is an ADR? An Architecture Decision Record captures: - **Context**: Why we needed to make a decision - **Decision**: What we decided - **Consequences**: What happens as a result ### 2. When to Write an ADR | Write ADR | Skip ADR | | -------------------------- | ---------------------- | | New framework adoption | Minor version upgrades | | Database technology choice | Bug fixes | | API design patterns | Implementation details | | Security architecture | Routine maintenance | | Integration patterns | Configuration changes | ### 3. ADR Lifecycle ``` Proposed β†’ Accepted β†’ Deprecated β†’ Superseded ↓ Rejected ``` ## Templates ### Template 1: Standard ADR (MADR Format) ```markdown # ADR-0001: Use PostgreSQL as Primary Database ## Status Accepted ## Context We need to select a primary database for our new e-commerce platform. The system will handle: - ~10,000 concurrent users - Complex product catalog with hierarchical categories - Transaction processing for orders and payments - Full-text search for products - Geospatial queries for store locator The team has experience with MySQL, PostgreSQL, and MongoDB. We need ACID compliance for financial transactions. ## Decision Drivers - **Must have ACID compliance** for payment processing - **Must support complex queries** for reporting - **Should support full-text search** to reduce infrastructure complexity - **Should have good JSON support** for flexible product attributes - **Team familiarity** reduces onboarding time ## Considered Options ### Option 1: PostgreSQL - **Pros**: ACID compliant, excellent JSON support (JSONB), built-in full-text search, PostGIS for geospatial, team has experience - **Cons**: Slightly more complex replication setup than MySQL ### Option 2: MySQL - **Pros**: Very familiar to team, simple replication, large community - **Cons**: Weaker JSON support, no built-in full-text search (need Elasticsearch), no geospatial without extensions ### Option 3: MongoDB - **Pros**: Flexible schema, native JSON, horizontal scaling - **Cons**: No ACID for multi-document transactions (at decision time), team has limited experience, requires schema design discipline ## Decision We will use **PostgreSQL 15** as our primary database. ## Rationale PostgreSQL provides the best balance of: 1. **ACID compliance** essential for e-commerce transactions 2. **Built-in capabilities** (full-text search, JSONB, PostGIS) reduce infrastructure complexity 3. **Team familiarity** with SQL databases reduces learning curve 4. **Mature ecosystem** with excellent tooling and community support The slight complexity in replication is outweighed by the reduction in additional services (no separate Elasticsearch needed). ## Consequences ### Positive - Single database handles transactions, search, and geospatial queries - Reduced operational complexity (fewer services to manage) - Strong consistency guarantees for financial data - Team can leverage existing SQL expertise ### Negative - Need to learn PostgreSQL-specific features (JSONB, full-text search syntax) - Vertical scaling limits may require read replicas sooner - Some team members need PostgreSQL-specific training ### Risks - Full-text search may not scale as well as dedicated search engines - Mitigation: Design for potential Elasticsearch addition if needed ## Implementation Notes - Use JSONB for flexible product attributes - Implement connection pooling with PgBouncer - Set up streaming replication for read replicas - Use pg_trgm extension for fuzzy search ## Related Decisions - ADR-0002: Caching Strategy (Redis) - complements database choice - ADR-0005: Search Architecture - may supersede if Elasticsearch needed ## References - [PostgreSQL JSON Documentation](https://www.postgresql.org/docs/current/datatype-json.html) - [PostgreSQL Full Text Search](https://www.postgresql.org/docs/current/textsearch.html) - Internal: Performance benchmarks in `/docs/benchmarks/database-comparison.md` ``` ### Template 2: Lightweight ADR ```markdown # ADR-0012: Adopt TypeScript for Frontend Development **Status**: Accepted **Date**: 2024-01-15 **Deciders**: @alice, @bob, @charlie ## Context Our React codebase has grown to 50+ components with increasing bug reports related to prop type mismatches and undefined errors. PropTypes provide runtime-only checking. ## Decision Adopt TypeScript for all new frontend code. Migrate existing code incrementally. ## Consequences **Good**: Catch type errors at compile time, better IDE support, self-documenting code. **Bad**: Learning curve for team, initial slowdown, build complexity increase. **Mitigations**: TypeScript training sessions, allow gradual adoption with `allowJs: true`. ``` ### Template 3: Y-Statement Format ```markdown # ADR-0015: API Gateway Selection In the context of **building a microservices architecture**, facing **the need for centralized API management, authentication, and rate limiting**, we decided for **Kong Gateway** and against **AWS API Gateway and custom Nginx solution**, to achieve **vendor independence, plugin extensibility, and team familiarity with Lua**, accepting that **we need to manage Kong infrastructure ourselves**. ``` ### Template 4: ADR for Deprecation ```markdown # ADR-0020: Deprecate MongoDB in Favor of PostgreSQL ## Status Accepted (Supersedes ADR-0003) ## Context ADR-0003 (2021) chose MongoDB for user profile storage due to schema flexibility needs. Since then: - MongoDB's multi-document transactions remain problematic for our use case - Our schema has stabilized and rarely changes - We now have PostgreSQL expertise from other services - Maintaining two databases increases operational burden ## Decision Deprecate MongoDB and migrate user profiles to PostgreSQL. ## Migration Plan 1. **Phase 1** (Week 1-2): Create PostgreSQL schema, dual-write enabled 2. **Phase 2** (Week 3-4): Backfill historical data, validate consistency 3. **Phase 3** (Week 5): Switch reads to PostgreSQL, monitor 4. **Phase 4** (Week 6): Remove MongoDB writes, decommission ## Consequences ### Positive - Single database technology reduces operational complexity - ACID transactions for user data - Team can focus PostgreSQL expertise ### Negative - Migration effort (~4 weeks) - Risk of data issues during migration - Lose some schema flexibility ## Lessons Learned Document from ADR-0003 experience: - Schema flexibility benefits were overestimated - Operational cost of multiple databases was underestimated - Consider long-term maintenance in technology decisions ``` ### Template 5: Request for Comments (RFC) Style ```markdown # RFC-0025: Adopt Event Sourcing for Order Management ## Summary Propose adopting event sourcing pattern for the order management domain to improve auditability, enable temporal queries, and support business analytics. ## Motivation Current challenges: 1. Audit requirements need complete order history 2. "What was the order state at time X?" queries are impossible 3. Analytics team needs event stream for real-time dashboards 4. Order state reconstruction for customer support is manual ## Detailed Design ### Event Store ``` OrderCreated { orderId, customerId, items[], timestamp } OrderItemAdded { orderId, item, timestamp } OrderItemRemoved { orderId, itemId, timestamp } PaymentReceived { orderId, amount, paymentId, timestamp } OrderShipped { orderId, trackingNumber, timestamp } ``` ### Projections - **CurrentOrderState**: Materialized view for queries - **OrderHistory**: Complete timeline for audit - **DailyOrderMetrics**: Analytics aggregation ### Technology - Event Store: EventStoreDB (purpose-built, handles projections) - Alternative considered: Kafka + custom projection service ## Drawbacks - Learning curve for team - Increased complexity vs. CRUD - Need to design events carefully (immutable once stored) - Storage growth (events never deleted) ## Alternatives 1. **Audit tables**: Simpler but doesn't enable temporal queries 2. **CDC from existing DB**: Complex, doesn't change data model 3. **Hybrid**: Event source only for order state changes ## Unresolved Questions - [ ] Event schema versioning strategy - [ ] Retention policy for events - [ ] Snapshot frequency for performance ## Implementation Plan 1. Prototype with single order type (2 weeks) 2. Team training on event sourcing (1 week) 3. Full implementation and migration (4 weeks) 4. Monitoring and optimization (ongoing) ## References - [Event Sourcing by Martin Fowler](https://martinfowler.com/eaaDev/EventSourcing.html) - [EventStoreDB Documentation](https://www.eventstore.com/docs) ``` ## ADR Management ### Directory Structure ``` docs/ β”œβ”€β”€ adr/ β”‚ β”œβ”€β”€ README.md # Index and guidelines β”‚ β”œβ”€β”€ template.md # Team's ADR template β”‚ β”œβ”€β”€ 0001-use-postgresql.md β”‚ β”œβ”€β”€ 0002-caching-strategy.md β”‚ β”œβ”€β”€ 0003-mongodb-user-profiles.md # [DEPRECATED] β”‚ └── 0020-deprecate-mongodb.md # Supersedes 0003 ``` ### ADR Index (README.md) ```markdown # Architecture Decision Records This directory contains Architecture Decision Records (ADRs) for [Project Name]. ## Index | ADR | Title | Status | Date | | ------------------------------------- | ---------------------------------- | ---------- | ---------- | | [0001](0001-use-postgresql.md) | Use PostgreSQL as Primary Database | Accepted | 2024-01-10 | | [0002](0002-caching-strategy.md) | Caching Strategy with Redis | Accepted | 2024-01-12 | | [0003](0003-mongodb-user-profiles.md) | MongoDB for User Profiles | Deprecated | 2023-06-15 | | [0020](0020-deprecate-mongodb.md) | Deprecate MongoDB | Accepted | 2024-01-15 | ## Creating a New ADR 1. Copy `template.md` to `NNNN-title-with-dashes.md` 2. Fill in the template 3. Submit PR for review 4. Update this index after approval ## ADR Status - **Proposed**: Under discussion - **Accepted**: Decision made, implementing - **Deprecated**: No longer relevant - **Superseded**: Replaced by another ADR - **Rejected**: Considered but not adopted ``` ### Automation (adr-tools) ```bash # Install adr-tools brew install adr-tools # Initialize ADR directory adr init docs/adr # Create new ADR adr new "Use PostgreSQL as Primary Database" # Supersede an ADR adr new -s 3 "Deprecate MongoDB in Favor of PostgreSQL" # Generate table of contents adr generate toc > docs/adr/README.md # Link related ADRs adr link 2 "Complements" 1 "Is complemented by" ``` ## Review Process ```markdown ## ADR Review Checklist ### Before Submission - [ ] Context clearly explains the problem - [ ] All viable options considered - [ ] Pros/cons balanced and honest - [ ] Consequences (positive and negative) documented - [ ] Related ADRs linked ### During Review - [ ] At least 2 senior engineers reviewed - [ ] Affected teams consulted - [ ] Security implications considered - [ ] Cost implications documented - [ ] Reversibility assessed ### After Acceptance - [ ] ADR index updated - [ ] Team notified - [ ] Implementation tickets created - [ ] Related documentation updated ``` ## Best Practices ### Do's - **Write ADRs early** - Before implementation starts - **Keep them short** - 1-2 pages maximum - **Be honest about trade-offs** - Include real cons - **Link related decisions** - Build decision graph - **Update status** - Deprecate when superseded ### Don'ts - **Don't change accepted ADRs** - Write new ones to supersede - **Don't skip context** - Future readers need background - **Don't hide failures** - Rejected decisions are valuable - **Don't be vague** - Specific decisions, specific consequences - **Don't forget implementation** - ADR without action is waste ## Resources - [Documenting Architecture Decisions (Michael Nygard)](https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions) - [MADR Template](https://adr.github.io/madr/) - [ADR GitHub Organization](https://adr.github.io/) - [adr-tools](https://github.com/npryce/adr-tools)
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

employment-contract-templates

Create employment contracts, offer letters, and HR policy documents

coding
⭐1
# Employment Contract Templates Templates and patterns for creating legally sound employment documentation including contracts, offer letters, and HR policies. ## When to Use This Skill - Drafting employment contracts - Creating offer letters - Writing employee handbooks - Developing HR policies - Standardizing employment documentation - Onboarding documentation ## Core Concepts ### 1. Employment Document Types | Document | Purpose | When Used | | ----------------------- | ----------------------- | ------------- | | **Offer Letter** | Initial job offer | Pre-hire | | **Employment Contract** | Formal agreement | Hire | | **Employee Handbook** | Policies & procedures | Onboarding | | **NDA** | Confidentiality | Before access | | **Non-Compete** | Competition restriction | Hire/Exit | ### 2. Key Legal Considerations ``` Employment Relationship: β”œβ”€β”€ At-Will vs. Contract β”œβ”€β”€ Employee vs. Contractor β”œβ”€β”€ Full-Time vs. Part-Time β”œβ”€β”€ Exempt vs. Non-Exempt └── Jurisdiction-Specific Requirements ``` **DISCLAIMER: These templates are for informational purposes only and do not constitute legal advice. Consult with qualified legal counsel before using any employment documents.** ## Templates ### Template 1: Offer Letter ```markdown # EMPLOYMENT OFFER LETTER [Company Letterhead] Date: [DATE] [Candidate Name] [Address] [City, State ZIP] Dear [Candidate Name], We are pleased to extend an offer of employment for the position of [JOB TITLE] at [COMPANY NAME]. We believe your skills and experience will be valuable additions to our team. ## Position Details **Title:** [Job Title] **Department:** [Department] **Reports To:** [Manager Name/Title] **Location:** [Office Location / Remote] **Start Date:** [Proposed Start Date] **Employment Type:** [Full-Time/Part-Time], [Exempt/Non-Exempt] ## Compensation **Base Salary:** $[AMOUNT] per [year/hour], paid [bi-weekly/semi-monthly/monthly] **Bonus:** [Eligible for annual bonus of up to X% based on company and individual performance / Not applicable] **Equity:** [X shares of stock options vesting over 4 years with 1-year cliff / Not applicable] ## Benefits You will be eligible for our standard benefits package, including: - Health insurance (medical, dental, vision) effective [date] - 401(k) with [X]% company match - [x] days paid time off per year - [x] paid holidays - [Other benefits] Full details will be provided during onboarding. ## Contingencies This offer is contingent upon: - Successful completion of background check - Verification of your right to work in [Country] - Execution of required employment documents including: - Confidentiality Agreement - [Non-Compete Agreement, if applicable] - [IP Assignment Agreement] ## At-Will Employment Please note that employment with [Company Name] is at-will. This means that either you or the Company may terminate the employment relationship at any time, with or without cause or notice. This offer letter does not constitute a contract of employment for any specific period. ## Acceptance To accept this offer, please sign below and return by [DEADLINE DATE]. This offer will expire if not accepted by that date. We are excited about the possibility of you joining our team. If you have any questions, please contact [HR Contact] at [email/phone]. Sincerely, --- [Hiring Manager Name] [Title] [Company Name] --- ## ACCEPTANCE I accept this offer of employment and agree to the terms stated above. Signature: ************\_************ Printed Name: ************\_************ Date: ************\_************ Anticipated Start Date: ************\_************ ``` ### Template 2: Employment Agreement (Contract Position) ```markdown # EMPLOYMENT AGREEMENT This Employment Agreement ("Agreement") is entered into as of [DATE] ("Effective Date") by and between: **Employer:** [COMPANY LEGAL NAME], a [State] [corporation/LLC] with principal offices at [Address] ("Company") **Employee:** [EMPLOYEE NAME], an individual residing at [Address] ("Employee") ## 1. EMPLOYMENT 1.1 **Position.** The Company agrees to employ Employee as [JOB TITLE], reporting to [Manager Title]. Employee accepts such employment subject to the terms of this Agreement. 1.2 **Duties.** Employee shall perform duties consistent with their position, including but not limited to: - [Primary duty 1] - [Primary duty 2] - [Primary duty 3] - Other duties as reasonably assigned 1.3 **Best Efforts.** Employee agrees to devote their full business time, attention, and best efforts to the Company's business during employment. 1.4 **Location.** Employee's primary work location shall be [Location/Remote]. [Travel requirements, if any.] ## 2. TERM 2.1 **Employment Period.** This Agreement shall commence on [START DATE] and continue until terminated as provided herein. 2.2 **At-Will Employment.** [FOR AT-WILL STATES] Notwithstanding anything herein, employment is at-will and may be terminated by either party at any time, with or without cause or notice. [OR FOR FIXED TERM:] 2.2 **Fixed Term.** This Agreement is for a fixed term of [X] months/years, ending on [END DATE], unless terminated earlier as provided herein or extended by mutual written agreement. ## 3. COMPENSATION 3.1 **Base Salary.** Employee shall receive a base salary of $[AMOUNT] per year, payable in accordance with the Company's standard payroll practices, subject to applicable withholdings. 3.2 **Bonus.** Employee may be eligible for an annual discretionary bonus of up to [X]% of base salary, based on [criteria]. Bonus payments are at Company's sole discretion and require active employment at payment date. 3.3 **Equity.** [If applicable] Subject to Board approval and the Company's equity incentive plan, Employee shall be granted [X shares/options] under the terms of a separate Stock Option Agreement. 3.4 **Benefits.** Employee shall be entitled to participate in benefit plans offered to similarly situated employees, subject to plan terms and eligibility requirements. 3.5 **Expenses.** Company shall reimburse Employee for reasonable business expenses incurred in accordance with Company policy. ## 4. CONFIDENTIALITY 4.1 **Confidential Information.** Employee acknowledges access to confidential and proprietary information including: trade secrets, business plans, customer lists, financial data, technical information, and other non-public information ("Confidential Information"). 4.2 **Non-Disclosure.** During and after employment, Employee shall not disclose, use, or permit use of any Confidential Information except as required for their duties or with prior written consent. 4.3 **Return of Materials.** Upon termination, Employee shall immediately return all Company property and Confidential Information in any form. 4.4 **Survival.** Confidentiality obligations survive termination indefinitely for trade secrets and for [3] years for other Confidential Information. ## 5. INTELLECTUAL PROPERTY 5.1 **Work Product.** All inventions, discoveries, works, and developments created by Employee during employment, relating to Company's business, or using Company resources ("Work Product") shall be Company's sole property. 5.2 **Assignment.** Employee hereby assigns to Company all rights in Work Product, including all intellectual property rights. 5.3 **Assistance.** Employee agrees to execute documents and take actions necessary to perfect Company's rights in Work Product. 5.4 **Prior Inventions.** Attached as Exhibit A is a list of any prior inventions that Employee wishes to exclude from this Agreement. ## 6. NON-COMPETITION AND NON-SOLICITATION [NOTE: Enforceability varies by jurisdiction. Consult local counsel.] 6.1 **Non-Competition.** During employment and for [12] months after termination, Employee shall not, directly or indirectly, engage in any business competitive with Company's business within [Geographic Area]. 6.2 **Non-Solicitation of Customers.** During employment and for [12] months after termination, Employee shall not solicit any customer of the Company for competing products or services. 6.3 **Non-Solicitation of Employees.** During employment and for [12] months after termination, Employee shall not recruit or solicit any Company employee to leave Company employment. ## 7. TERMINATION 7.1 **By Company for Cause.** Company may terminate immediately for Cause, defined as: (a) Material breach of this Agreement (b) Conviction of a felony (c) Fraud, dishonesty, or gross misconduct (d) Failure to perform duties after written notice and cure period 7.2 **By Company Without Cause.** Company may terminate without Cause upon [30] days written notice. 7.3 **By Employee.** Employee may terminate upon [30] days written notice. 7.4 **Severance.** [If applicable] Upon termination without Cause, Employee shall receive [X] weeks base salary as severance, contingent upon execution of a release agreement. 7.5 **Effect of Termination.** Upon termination: - All compensation earned through termination date shall be paid - Unvested equity shall be forfeited - Benefits terminate per plan terms - Sections 4, 5, 6, 8, and 9 survive termination ## 8. GENERAL PROVISIONS 8.1 **Entire Agreement.** This Agreement constitutes the entire agreement and supersedes all prior negotiations, representations, and agreements. 8.2 **Amendments.** This Agreement may be amended only by written agreement signed by both parties. 8.3 **Governing Law.** This Agreement shall be governed by the laws of [State], without regard to conflicts of law principles. 8.4 **Dispute Resolution.** [Arbitration clause or jurisdiction selection] 8.5 **Severability.** If any provision is unenforceable, it shall be modified to the minimum extent necessary, and remaining provisions shall remain in effect. 8.6 **Notices.** Notices shall be in writing and delivered to addresses above. 8.7 **Assignment.** Employee may not assign this Agreement. Company may assign to a successor. 8.8 **Waiver.** Failure to enforce any provision shall not constitute waiver. ## 9. ACKNOWLEDGMENTS Employee acknowledges: - Having read and understood this Agreement - Having opportunity to consult with counsel - Agreeing to all terms voluntarily --- IN WITNESS WHEREOF, the parties have executed this Agreement as of the Effective Date. **[COMPANY NAME]** By: ************\_************ Name: [Authorized Signatory] Title: [Title] Date: ************\_************ **EMPLOYEE** Signature: ************\_************ Name: [Employee Name] Date: ************\_************ --- ## EXHIBIT A: PRIOR INVENTIONS [Employee to list any prior inventions, if any, or write "None"] --- ``` ### Template 3: Employee Handbook Policy Section ```markdown # EMPLOYEE HANDBOOK - POLICY SECTION ## EMPLOYMENT POLICIES ### Equal Employment Opportunity [Company Name] is an equal opportunity employer. We do not discriminate based on race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, veteran status, or any other protected characteristic. This policy applies to all employment practices including: - Recruitment and hiring - Compensation and benefits - Training and development - Promotions and transfers - Termination ### Anti-Harassment Policy [Company Name] is committed to providing a workplace free from harassment. Harassment based on any protected characteristic is strictly prohibited. **Prohibited Conduct Includes:** - Unwelcome sexual advances or requests for sexual favors - Offensive comments, jokes, or slurs - Physical conduct such as assault or unwanted touching - Visual conduct such as displaying offensive images - Threatening, intimidating, or hostile acts **Reporting Procedure:** 1. Report to your manager, HR, or any member of leadership 2. Reports may be made verbally or in writing 3. Anonymous reports are accepted via [hotline/email] **Investigation:** All reports will be promptly investigated. Retaliation against anyone who reports harassment is strictly prohibited and will result in disciplinary action up to termination. ### Work Hours and Attendance **Standard Hours:** [8:00 AM - 5:00 PM, Monday through Friday] **Core Hours:** [10:00 AM - 3:00 PM] - Employees expected to be available **Flexible Work:** [Policy on remote work, flexible scheduling] **Attendance Expectations:** - Notify your manager as soon as possible if you will be absent - Excessive unexcused absences may result in disciplinary action - [x] unexcused absences in [Y] days considered excessive ### Paid Time Off (PTO) **PTO Accrual:** | Years of Service | Annual PTO Days | |------------------|-----------------| | 0-2 years | 15 days | | 3-5 years | 20 days | | 6+ years | 25 days | **PTO Guidelines:** - PTO accrues per pay period - Maximum accrual: [X] days (use it or lose it after) - Request PTO at least [2] weeks in advance - Manager approval required - PTO may not be taken during [blackout periods] ### Sick Leave - [x] days sick leave per year - May be used for personal illness or family member care - Doctor's note required for absences exceeding [3] days ### Holidays The following paid holidays are observed: - New Year's Day - Martin Luther King Jr. Day - Presidents Day - Memorial Day - Independence Day - Labor Day - Thanksgiving Day - Day after Thanksgiving - Christmas Day - [Floating holiday] ### Code of Conduct All employees are expected to: - Act with integrity and honesty - Treat colleagues, customers, and partners with respect - Protect company confidential information - Avoid conflicts of interest - Comply with all laws and regulations - Report any violations of this code **Violations may result in disciplinary action up to and including termination.** ### Technology and Communication **Acceptable Use:** - Company technology is for business purposes - Limited personal use is permitted if it doesn't interfere with work - No illegal activities or viewing inappropriate content **Monitoring:** - Company reserves the right to monitor company systems - Employees should have no expectation of privacy on company devices **Security:** - Use strong passwords and enable 2FA - Report security incidents immediately - Lock devices when unattended ### Social Media Policy **Personal Social Media:** - Clearly state opinions are your own, not the company's - Do not share confidential company information - Be respectful and professional **Company Social Media:** - Only authorized personnel may post on behalf of the company - Follow brand guidelines - Escalate negative comments to [Marketing/PR] --- ## ACKNOWLEDGMENT I acknowledge that I have received a copy of the Employee Handbook and understand that: 1. I am responsible for reading and understanding its contents 2. The handbook does not create a contract of employment 3. Policies may be changed at any time at the company's discretion 4. Employment is at-will [if applicable] I agree to abide by the policies and procedures outlined in this handbook. Employee Signature: ************\_************ Employee Name (Print): ************\_************ Date: ************\_************ ``` ## Best Practices ### Do's - **Consult legal counsel** - Employment law varies by jurisdiction - **Keep copies signed** - Document all agreements - **Update regularly** - Laws and policies change - **Be clear and specific** - Avoid ambiguity - **Train managers** - On policies and procedures ### Don'ts - **Don't use generic templates** - Customize for your jurisdiction - **Don't make promises** - That could create implied contracts - **Don't discriminate** - In language or application - **Don't forget at-will language** - Where applicable - **Don't skip review** - Have legal counsel review all documents ## Resources - [SHRM Employment Templates](https://www.shrm.org/) - [Department of Labor](https://www.dol.gov/) - [EEOC Guidance](https://www.eeoc.gov/) - State-specific labor departments
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

fastapi-templates

Create production-ready FastAPI projects with async patterns,

coding
⭐1
# FastAPI Project Templates Production-ready FastAPI project structures with async patterns, dependency injection, middleware, and best practices for building high-performance APIs. ## When to Use This Skill - Starting new FastAPI projects from scratch - Implementing async REST APIs with Python - Building high-performance web services and microservices - Creating async applications with PostgreSQL, MongoDB - Setting up API projects with proper structure and testing ## Core Concepts ### 1. Project Structure **Recommended Layout:** ``` app/ β”œβ”€β”€ api/ # API routes β”‚ β”œβ”€β”€ v1/ β”‚ β”‚ β”œβ”€β”€ endpoints/ β”‚ β”‚ β”‚ β”œβ”€β”€ users.py β”‚ β”‚ β”‚ β”œβ”€β”€ auth.py β”‚ β”‚ β”‚ └── items.py β”‚ β”‚ └── router.py β”‚ └── dependencies.py # Shared dependencies β”œβ”€β”€ core/ # Core configuration β”‚ β”œβ”€β”€ config.py β”‚ β”œβ”€β”€ security.py β”‚ └── database.py β”œβ”€β”€ models/ # Database models β”‚ β”œβ”€β”€ user.py β”‚ └── item.py β”œβ”€β”€ schemas/ # Pydantic schemas β”‚ β”œβ”€β”€ user.py β”‚ └── item.py β”œβ”€β”€ services/ # Business logic β”‚ β”œβ”€β”€ user_service.py β”‚ └── auth_service.py β”œβ”€β”€ repositories/ # Data access β”‚ β”œβ”€β”€ user_repository.py β”‚ └── item_repository.py └── main.py # Application entry ``` ### 2. Dependency Injection FastAPI's built-in DI system using `Depends`: - Database session management - Authentication/authorization - Shared business logic - Configuration injection ### 3. Async Patterns Proper async/await usage: - Async route handlers - Async database operations - Async background tasks - Async middleware ## Implementation Patterns ### Pattern 1: Complete FastAPI Application ```python # main.py from fastapi import FastAPI, Depends from fastapi.middleware.cors import CORSMiddleware from contextlib import asynccontextmanager @asynccontextmanager async def lifespan(app: FastAPI): """Application lifespan events.""" # Startup await database.connect() yield # Shutdown await database.disconnect() app = FastAPI( title="API Template", version="1.0.0", lifespan=lifespan ) # CORS middleware app.add_middleware( CORSMiddleware, allow_origins=["*"], allow_credentials=True, allow_methods=["*"], allow_headers=["*"], ) # Include routers from app.api.v1.router import api_router app.include_router(api_router, prefix="/api/v1") # core/config.py from pydantic_settings import BaseSettings from functools import lru_cache class Settings(BaseSettings): """Application settings.""" DATABASE_URL: str SECRET_KEY: str ACCESS_TOKEN_EXPIRE_MINUTES: int = 30 API_V1_STR: str = "/api/v1" class Config: env_file = ".env" @lru_cache() def get_settings() -> Settings: return Settings() # core/database.py from sqlalchemy.ext.asyncio import create_async_engine, AsyncSession from sqlalchemy.ext.declarative import declarative_base from sqlalchemy.orm import sessionmaker from app.core.config import get_settings settings = get_settings() engine = create_async_engine( settings.DATABASE_URL, echo=True, future=True ) AsyncSessionLocal = sessionmaker( engine, class_=AsyncSession, expire_on_commit=False ) Base = declarative_base() async def get_db() -> AsyncSession: """Dependency for database session.""" async with AsyncSessionLocal() as session: try: yield session await session.commit() except Exception: await session.rollback() raise finally: await session.close() ``` ### Pattern 2: CRUD Repository Pattern ```python # repositories/base_repository.py from typing import Generic, TypeVar, Type, Optional, List from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy import select from pydantic import BaseModel ModelType = TypeVar("ModelType") CreateSchemaType = TypeVar("CreateSchemaType", bound=BaseModel) UpdateSchemaType = TypeVar("UpdateSchemaType", bound=BaseModel) class BaseRepository(Generic[ModelType, CreateSchemaType, UpdateSchemaType]): """Base repository for CRUD operations.""" def __init__(self, model: Type[ModelType]): self.model = model async def get(self, db: AsyncSession, id: int) -> Optional[ModelType]: """Get by ID.""" result = await db.execute( select(self.model).where(self.model.id == id) ) return result.scalars().first() async def get_multi( self, db: AsyncSession, skip: int = 0, limit: int = 100 ) -> List[ModelType]: """Get multiple records.""" result = await db.execute( select(self.model).offset(skip).limit(limit) ) return result.scalars().all() async def create( self, db: AsyncSession, obj_in: CreateSchemaType ) -> ModelType: """Create new record.""" db_obj = self.model(**obj_in.dict()) db.add(db_obj) await db.flush() await db.refresh(db_obj) return db_obj async def update( self, db: AsyncSession, db_obj: ModelType, obj_in: UpdateSchemaType ) -> ModelType: """Update record.""" update_data = obj_in.dict(exclude_unset=True) for field, value in update_data.items(): setattr(db_obj, field, value) await db.flush() await db.refresh(db_obj) return db_obj async def delete(self, db: AsyncSession, id: int) -> bool: """Delete record.""" obj = await self.get(db, id) if obj: await db.delete(obj) return True return False # repositories/user_repository.py from app.repositories.base_repository import BaseRepository from app.models.user import User from app.schemas.user import UserCreate, UserUpdate class UserRepository(BaseRepository[User, UserCreate, UserUpdate]): """User-specific repository.""" async def get_by_email(self, db: AsyncSession, email: str) -> Optional[User]: """Get user by email.""" result = await db.execute( select(User).where(User.email == email) ) return result.scalars().first() async def is_active(self, db: AsyncSession, user_id: int) -> bool: """Check if user is active.""" user = await self.get(db, user_id) return user.is_active if user else False user_repository = UserRepository(User) ``` ### Pattern 3: Service Layer ```python # services/user_service.py from typing import Optional from sqlalchemy.ext.asyncio import AsyncSession from app.repositories.user_repository import user_repository from app.schemas.user import UserCreate, UserUpdate, User from app.core.security import get_password_hash, verify_password class UserService: """Business logic for users.""" def __init__(self): self.repository = user_repository async def create_user( self, db: AsyncSession, user_in: UserCreate ) -> User: """Create new user with hashed password.""" # Check if email exists existing = await self.repository.get_by_email(db, user_in.email) if existing: raise ValueError("Email already registered") # Hash password user_in_dict = user_in.dict() user_in_dict["hashed_password"] = get_password_hash(user_in_dict.pop("password")) # Create user user = await self.repository.create(db, UserCreate(**user_in_dict)) return user async def authenticate( self, db: AsyncSession, email: str, password: str ) -> Optional[User]: """Authenticate user.""" user = await self.repository.get_by_email(db, email) if not user: return None if not verify_password(password, user.hashed_password): return None return user async def update_user( self, db: AsyncSession, user_id: int, user_in: UserUpdate ) -> Optional[User]: """Update user.""" user = await self.repository.get(db, user_id) if not user: return None if user_in.password: user_in_dict = user_in.dict(exclude_unset=True) user_in_dict["hashed_password"] = get_password_hash( user_in_dict.pop("password") ) user_in = UserUpdate(**user_in_dict) return await self.repository.update(db, user, user_in) user_service = UserService() ``` ### Pattern 4: API Endpoints with Dependencies ```python # api/v1/endpoints/users.py from fastapi import APIRouter, Depends, HTTPException, status from sqlalchemy.ext.asyncio import AsyncSession from typing import List from app.core.database import get_db from app.schemas.user import User, UserCreate, UserUpdate from app.services.user_service import user_service from app.api.dependencies import get_current_user router = APIRouter() @router.post("/", response_model=User, status_code=status.HTTP_201_CREATED) async def create_user( user_in: UserCreate, db: AsyncSession = Depends(get_db) ): """Create new user.""" try: user = await user_service.create_user(db, user_in) return user except ValueError as e: raise HTTPException(status_code=400, detail=str(e)) @router.get("/me", response_model=User) async def read_current_user( current_user: User = Depends(get_current_user) ): """Get current user.""" return current_user @router.get("/{user_id}", response_model=User) async def read_user( user_id: int, db: AsyncSession = Depends(get_db), current_user: User = Depends(get_current_user) ): """Get user by ID.""" user = await user_service.repository.get(db, user_id) if not user: raise HTTPException(status_code=404, detail="User not found") return user @router.patch("/{user_id}", response_model=User) async def update_user( user_id: int, user_in: UserUpdate, db: AsyncSession = Depends(get_db), current_user: User = Depends(get_current_user) ): """Update user.""" if current_user.id != user_id: raise HTTPException(status_code=403, detail="Not authorized") user = await user_service.update_user(db, user_id, user_in) if not user: raise HTTPException(status_code=404, detail="User not found") return user @router.delete("/{user_id}", status_code=status.HTTP_204_NO_CONTENT) async def delete_user( user_id: int, db: AsyncSession = Depends(get_db), current_user: User = Depends(get_current_user) ): """Delete user.""" if current_user.id != user_id: raise HTTPException(status_code=403, detail="Not authorized") deleted = await user_service.repository.delete(db, user_id) if not deleted: raise HTTPException(status_code=404, detail="User not found") ``` ### Pattern 5: Authentication & Authorization ```python # core/security.py from datetime import datetime, timedelta from typing import Optional from jose import JWTError, jwt from passlib.context import CryptContext from app.core.config import get_settings settings = get_settings() pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto") ALGORITHM = "HS256" def create_access_token(data: dict, expires_delta: Optional[timedelta] = None): """Create JWT access token.""" to_encode = data.copy() if expires_delta: expire = datetime.utcnow() + expires_delta else: expire = datetime.utcnow() + timedelta(minutes=15) to_encode.update({"exp": expire}) encoded_jwt = jwt.encode(to_encode, settings.SECRET_KEY, algorithm=ALGORITHM) return encoded_jwt def verify_password(plain_password: str, hashed_password: str) -> bool: """Verify password against hash.""" return pwd_context.verify(plain_password, hashed_password) def get_password_hash(password: str) -> str: """Hash password.""" return pwd_context.hash(password) # api/dependencies.py from fastapi import Depends, HTTPException, status from fastapi.security import OAuth2PasswordBearer from jose import JWTError, jwt from sqlalchemy.ext.asyncio import AsyncSession from app.core.database import get_db from app.core.security import ALGORITHM from app.core.config import get_settings from app.repositories.user_repository import user_repository oauth2_scheme = OAuth2PasswordBearer(tokenUrl=f"{settings.API_V1_STR}/auth/login") async def get_current_user( db: AsyncSession = Depends(get_db), token: str = Depends(oauth2_scheme) ): """Get current authenticated user.""" credentials_exception = HTTPException( status_code=status.HTTP_401_UNAUTHORIZED, detail="Could not validate credentials", headers={"WWW-Authenticate": "Bearer"}, ) try: payload = jwt.decode(token, settings.SECRET_KEY, algorithms=[ALGORITHM]) user_id: int = payload.get("sub") if user_id is None: raise credentials_exception except JWTError: raise credentials_exception user = await user_repository.get(db, user_id) if user is None: raise credentials_exception return user ``` ## Testing ```python # tests/conftest.py import pytest import asyncio from httpx import AsyncClient from sqlalchemy.ext.asyncio import create_async_engine, AsyncSession from sqlalchemy.orm import sessionmaker from app.main import app from app.core.database import get_db, Base TEST_DATABASE_URL = "sqlite+aiosqlite:///:memory:" @pytest.fixture(scope="session") def event_loop(): loop = asyncio.get_event_loop_policy().new_event_loop() yield loop loop.close() @pytest.fixture async def db_session(): engine = create_async_engine(TEST_DATABASE_URL, echo=True) async with engine.begin() as conn: await conn.run_sync(Base.metadata.create_all) AsyncSessionLocal = sessionmaker( engine, class_=AsyncSession, expire_on_commit=False ) async with AsyncSessionLocal() as session: yield session @pytest.fixture async def client(db_session): async def override_get_db(): yield db_session app.dependency_overrides[get_db] = override_get_db async with AsyncClient(app=app, base_url="http://test") as client: yield client # tests/test_users.py import pytest @pytest.mark.asyncio async def test_create_user(client): response = await client.post( "/api/v1/users/", json={ "email": "test@example.com", "password": "testpass123", "name": "Test User" } ) assert response.status_code == 201 data = response.json() assert data["email"] == "test@example.com" assert "id" in data ``` ## Resources - **references/fastapi-architecture.md**: Detailed architecture guide - **references/async-best-practices.md**: Async/await patterns - **references/testing-strategies.md**: Comprehensive testing guide - **assets/project-template/**: Complete FastAPI project - **assets/docker-compose.yml**: Development environment setup ## Best Practices 1. **Async All The Way**: Use async for database, external APIs 2. **Dependency Injection**: Leverage FastAPI's DI system 3. **Repository Pattern**: Separate data access from business logic 4. **Service Layer**: Keep business logic out of routes 5. **Pydantic Schemas**: Strong typing for request/response 6. **Error Handling**: Consistent error responses 7. **Testing**: Test all layers independently ## Common Pitfalls - **Blocking Code in Async**: Using synchronous database drivers - **No Service Layer**: Business logic in route handlers - **Missing Type Hints**: Loses FastAPI's benefits - **Ignoring Sessions**: Not properly managing database sessions - **No Testing**: Skipping integration tests - **Tight Coupling**: Direct database access in routes
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

gitops-workflow

Implement GitOps workflows with ArgoCD and Flux for automated,

architecture
⭐1
# GitOps Workflow Complete guide to implementing GitOps workflows with ArgoCD and Flux for automated Kubernetes deployments. ## Purpose Implement declarative, Git-based continuous delivery for Kubernetes using ArgoCD or Flux CD, following OpenGitOps principles. ## When to Use This Skill - Set up GitOps for Kubernetes clusters - Automate application deployments from Git - Implement progressive delivery strategies - Manage multi-cluster deployments - Configure automated sync policies - Set up secret management in GitOps ## OpenGitOps Principles 1. **Declarative** - Entire system described declaratively 2. **Versioned and Immutable** - Desired state stored in Git 3. **Pulled Automatically** - Software agents pull desired state 4. **Continuously Reconciled** - Agents reconcile actual vs desired state ## ArgoCD Setup ### 1. Installation ```bash # Create namespace kubectl create namespace argocd # Install ArgoCD kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml # Get admin password kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d ``` **Reference:** See `references/argocd-setup.md` for detailed setup ### 2. Repository Structure ``` gitops-repo/ β”œβ”€β”€ apps/ β”‚ β”œβ”€β”€ production/ β”‚ β”‚ β”œβ”€β”€ app1/ β”‚ β”‚ β”‚ β”œβ”€β”€ kustomization.yaml β”‚ β”‚ β”‚ └── deployment.yaml β”‚ β”‚ └── app2/ β”‚ └── staging/ β”œβ”€β”€ infrastructure/ β”‚ β”œβ”€β”€ ingress-nginx/ β”‚ β”œβ”€β”€ cert-manager/ β”‚ └── monitoring/ └── argocd/ β”œβ”€β”€ applications/ └── projects/ ``` ### 3. Create Application ```yaml # argocd/applications/my-app.yaml apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: my-app namespace: argocd spec: project: default source: repoURL: https://github.com/org/gitops-repo targetRevision: main path: apps/production/my-app destination: server: https://kubernetes.default.svc namespace: production syncPolicy: automated: prune: true selfHeal: true syncOptions: - CreateNamespace=true ``` ### 4. App of Apps Pattern ```yaml apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: applications namespace: argocd spec: project: default source: repoURL: https://github.com/org/gitops-repo targetRevision: main path: argocd/applications destination: server: https://kubernetes.default.svc namespace: argocd syncPolicy: automated: {} ``` ## Flux CD Setup ### 1. Installation ```bash # Install Flux CLI curl -s https://fluxcd.io/install.sh | sudo bash # Bootstrap Flux flux bootstrap github \ --owner=org \ --repository=gitops-repo \ --branch=main \ --path=clusters/production \ --personal ``` ### 2. Create GitRepository ```yaml apiVersion: source.toolkit.fluxcd.io/v1 kind: GitRepository metadata: name: my-app namespace: flux-system spec: interval: 1m url: https://github.com/org/my-app ref: branch: main ``` ### 3. Create Kustomization ```yaml apiVersion: kustomize.toolkit.fluxcd.io/v1 kind: Kustomization metadata: name: my-app namespace: flux-system spec: interval: 5m path: ./deploy prune: true sourceRef: kind: GitRepository name: my-app ``` ## Sync Policies ### Auto-Sync Configuration **ArgoCD:** ```yaml syncPolicy: automated: prune: true # Delete resources not in Git selfHeal: true # Reconcile manual changes allowEmpty: false retry: limit: 5 backoff: duration: 5s factor: 2 maxDuration: 3m ``` **Flux:** ```yaml spec: interval: 1m prune: true wait: true timeout: 5m ``` **Reference:** See `references/sync-policies.md` ## Progressive Delivery ### Canary Deployment with ArgoCD Rollouts ```yaml apiVersion: argoproj.io/v1alpha1 kind: Rollout metadata: name: my-app spec: replicas: 5 strategy: canary: steps: - setWeight: 20 - pause: { duration: 1m } - setWeight: 50 - pause: { duration: 2m } - setWeight: 100 ``` ### Blue-Green Deployment ```yaml strategy: blueGreen: activeService: my-app previewService: my-app-preview autoPromotionEnabled: false ``` ## Secret Management ### External Secrets Operator ```yaml apiVersion: external-secrets.io/v1beta1 kind: ExternalSecret metadata: name: db-credentials spec: refreshInterval: 1h secretStoreRef: name: aws-secrets-manager kind: SecretStore target: name: db-credentials data: - secretKey: password remoteRef: key: prod/db/password ``` ### Sealed Secrets ```bash # Encrypt secret kubeseal --format yaml < secret.yaml > sealed-secret.yaml # Commit sealed-secret.yaml to Git ``` ## Best Practices 1. **Use separate repos or branches** for different environments 2. **Implement RBAC** for Git repositories 3. **Enable notifications** for sync failures 4. **Use health checks** for custom resources 5. **Implement approval gates** for production 6. **Keep secrets out of Git** (use External Secrets) 7. **Use App of Apps pattern** for organization 8. **Tag releases** for easy rollback 9. **Monitor sync status** with alerts 10. **Test changes** in staging first ## Troubleshooting **Sync failures:** ```bash argocd app get my-app argocd app sync my-app --prune ``` **Out of sync status:** ```bash argocd app diff my-app argocd app sync my-app --force ``` ## Related Skills - `k8s-manifest-generator` - For creating manifests - `helm-chart-scaffolding` - For packaging applications
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

k8s-manifest-generator

Create production-ready Kubernetes manifests for Deployments,

architecture
⭐1
# Kubernetes Manifest Generator Step-by-step guidance for creating production-ready Kubernetes manifests including Deployments, Services, ConfigMaps, Secrets, and PersistentVolumeClaims. ## Purpose This skill provides comprehensive guidance for generating well-structured, secure, and production-ready Kubernetes manifests following cloud-native best practices and Kubernetes conventions. ## When to Use This Skill Use this skill when you need to: - Create new Kubernetes Deployment manifests - Define Service resources for network connectivity - Generate ConfigMap and Secret resources for configuration management - Create PersistentVolumeClaim manifests for stateful workloads - Follow Kubernetes best practices and naming conventions - Implement resource limits, health checks, and security contexts - Design manifests for multi-environment deployments ## Step-by-Step Workflow ### 1. Gather Requirements **Understand the workload:** - Application type (stateless/stateful) - Container image and version - Environment variables and configuration needs - Storage requirements - Network exposure requirements (internal/external) - Resource requirements (CPU, memory) - Scaling requirements - Health check endpoints **Questions to ask:** - What is the application name and purpose? - What container image and tag will be used? - Does the application need persistent storage? - What ports does the application expose? - Are there any secrets or configuration files needed? - What are the CPU and memory requirements? - Does the application need to be exposed externally? ### 2. Create Deployment Manifest **Follow this structure:** ```yaml apiVersion: apps/v1 kind: Deployment metadata: name: <app-name> namespace: <namespace> labels: app: <app-name> version: <version> spec: replicas: 3 selector: matchLabels: app: <app-name> template: metadata: labels: app: <app-name> version: <version> spec: containers: - name: <container-name> image: <image>:<tag> ports: - containerPort: <port> name: http resources: requests: memory: "256Mi" cpu: "250m" limits: memory: "512Mi" cpu: "500m" livenessProbe: httpGet: path: /health port: http initialDelaySeconds: 30 periodSeconds: 10 readinessProbe: httpGet: path: /ready port: http initialDelaySeconds: 5 periodSeconds: 5 env: - name: ENV_VAR value: "value" envFrom: - configMapRef: name: <app-name>-config - secretRef: name: <app-name>-secret ``` **Best practices to apply:** - Always set resource requests and limits - Implement both liveness and readiness probes - Use specific image tags (never `:latest`) - Apply security context for non-root users - Use labels for organization and selection - Set appropriate replica count based on availability needs **Reference:** See `references/deployment-spec.md` for detailed deployment options ### 3. Create Service Manifest **Choose the appropriate Service type:** **ClusterIP (internal only):** ```yaml apiVersion: v1 kind: Service metadata: name: <app-name> namespace: <namespace> labels: app: <app-name> spec: type: ClusterIP selector: app: <app-name> ports: - name: http port: 80 targetPort: 8080 protocol: TCP ``` **LoadBalancer (external access):** ```yaml apiVersion: v1 kind: Service metadata: name: <app-name> namespace: <namespace> labels: app: <app-name> annotations: service.beta.kubernetes.io/aws-load-balancer-type: nlb spec: type: LoadBalancer selector: app: <app-name> ports: - name: http port: 80 targetPort: 8080 protocol: TCP ``` **Reference:** See `references/service-spec.md` for service types and networking ### 4. Create ConfigMap **For application configuration:** ```yaml apiVersion: v1 kind: ConfigMap metadata: name: <app-name>-config namespace: <namespace> data: APP_MODE: production LOG_LEVEL: info DATABASE_HOST: db.example.com # For config files app.properties: | server.port=8080 server.host=0.0.0.0 logging.level=INFO ``` **Best practices:** - Use ConfigMaps for non-sensitive data only - Organize related configuration together - Use meaningful names for keys - Consider using one ConfigMap per component - Version ConfigMaps when making changes **Reference:** See `assets/configmap-template.yaml` for examples ### 5. Create Secret **For sensitive data:** ```yaml apiVersion: v1 kind: Secret metadata: name: <app-name>-secret namespace: <namespace> type: Opaque stringData: DATABASE_PASSWORD: "changeme" API_KEY: "secret-api-key" # For certificate files tls.crt: | -----BEGIN CERTIFICATE----- ... -----END CERTIFICATE----- tls.key: | -----BEGIN PRIVATE KEY----- ... -----END PRIVATE KEY----- ``` **Security considerations:** - Never commit secrets to Git in plain text - Use Sealed Secrets, External Secrets Operator, or Vault - Rotate secrets regularly - Use RBAC to limit secret access - Consider using Secret type: `kubernetes.io/tls` for TLS secrets ### 6. Create PersistentVolumeClaim (if needed) **For stateful applications:** ```yaml apiVersion: v1 kind: PersistentVolumeClaim metadata: name: <app-name>-data namespace: <namespace> spec: accessModes: - ReadWriteOnce storageClassName: gp3 resources: requests: storage: 10Gi ``` **Mount in Deployment:** ```yaml spec: template: spec: containers: - name: app volumeMounts: - name: data mountPath: /var/lib/app volumes: - name: data persistentVolumeClaim: claimName: <app-name>-data ``` **Storage considerations:** - Choose appropriate StorageClass for performance needs - Use ReadWriteOnce for single-pod access - Use ReadWriteMany for multi-pod shared storage - Consider backup strategies - Set appropriate retention policies ### 7. Apply Security Best Practices **Add security context to Deployment:** ```yaml spec: template: spec: securityContext: runAsNonRoot: true runAsUser: 1000 fsGroup: 1000 seccompProfile: type: RuntimeDefault containers: - name: app securityContext: allowPrivilegeEscalation: false readOnlyRootFilesystem: true capabilities: drop: - ALL ``` **Security checklist:** - [ ] Run as non-root user - [ ] Drop all capabilities - [ ] Use read-only root filesystem - [ ] Disable privilege escalation - [ ] Set seccomp profile - [ ] Use Pod Security Standards ### 8. Add Labels and Annotations **Standard labels (recommended):** ```yaml metadata: labels: app.kubernetes.io/name: <app-name> app.kubernetes.io/instance: <instance-name> app.kubernetes.io/version: "1.0.0" app.kubernetes.io/component: backend app.kubernetes.io/part-of: <system-name> app.kubernetes.io/managed-by: kubectl ``` **Useful annotations:** ```yaml metadata: annotations: description: "Application description" contact: "team@example.com" prometheus.io/scrape: "true" prometheus.io/port: "9090" prometheus.io/path: "/metrics" ``` ### 9. Organize Multi-Resource Manifests **File organization options:** **Option 1: Single file with `---` separator** ```yaml # app-name.yaml --- apiVersion: v1 kind: ConfigMap ... --- apiVersion: v1 kind: Secret ... --- apiVersion: apps/v1 kind: Deployment ... --- apiVersion: v1 kind: Service ... ``` **Option 2: Separate files** ``` manifests/ β”œβ”€β”€ configmap.yaml β”œβ”€β”€ secret.yaml β”œβ”€β”€ deployment.yaml β”œβ”€β”€ service.yaml └── pvc.yaml ``` **Option 3: Kustomize structure** ``` base/ β”œβ”€β”€ kustomization.yaml β”œβ”€β”€ deployment.yaml β”œβ”€β”€ service.yaml └── configmap.yaml overlays/ β”œβ”€β”€ dev/ β”‚ └── kustomization.yaml └── prod/ └── kustomization.yaml ``` ### 10. Validate and Test **Validation steps:** ```bash # Dry-run validation kubectl apply -f manifest.yaml --dry-run=client # Server-side validation kubectl apply -f manifest.yaml --dry-run=server # Validate with kubeval kubeval manifest.yaml # Validate with kube-score kube-score score manifest.yaml # Check with kube-linter kube-linter lint manifest.yaml ``` **Testing checklist:** - [ ] Manifest passes dry-run validation - [ ] All required fields are present - [ ] Resource limits are reasonable - [ ] Health checks are configured - [ ] Security context is set - [ ] Labels follow conventions - [ ] Namespace exists or is created ## Common Patterns ### Pattern 1: Simple Stateless Web Application **Use case:** Standard web API or microservice **Components needed:** - Deployment (3 replicas for HA) - ClusterIP Service - ConfigMap for configuration - Secret for API keys - HorizontalPodAutoscaler (optional) **Reference:** See `assets/deployment-template.yaml` ### Pattern 2: Stateful Database Application **Use case:** Database or persistent storage application **Components needed:** - StatefulSet (not Deployment) - Headless Service - PersistentVolumeClaim template - ConfigMap for DB configuration - Secret for credentials ### Pattern 3: Background Job or Cron **Use case:** Scheduled tasks or batch processing **Components needed:** - CronJob or Job - ConfigMap for job parameters - Secret for credentials - ServiceAccount with RBAC ### Pattern 4: Multi-Container Pod **Use case:** Application with sidecar containers **Components needed:** - Deployment with multiple containers - Shared volumes between containers - Init containers for setup - Service (if needed) ## Templates The following templates are available in the `assets/` directory: - `deployment-template.yaml` - Standard deployment with best practices - `service-template.yaml` - Service configurations (ClusterIP, LoadBalancer, NodePort) - `configmap-template.yaml` - ConfigMap examples with different data types - `secret-template.yaml` - Secret examples (to be generated, not committed) - `pvc-template.yaml` - PersistentVolumeClaim templates ## Reference Documentation - `references/deployment-spec.md` - Detailed Deployment specification - `references/service-spec.md` - Service types and networking details ## Best Practices Summary 1. **Always set resource requests and limits** - Prevents resource starvation 2. **Implement health checks** - Ensures Kubernetes can manage your application 3. **Use specific image tags** - Avoid unpredictable deployments 4. **Apply security contexts** - Run as non-root, drop capabilities 5. **Use ConfigMaps and Secrets** - Separate config from code 6. **Label everything** - Enables filtering and organization 7. **Follow naming conventions** - Use standard Kubernetes labels 8. **Validate before applying** - Use dry-run and validation tools 9. **Version your manifests** - Keep in Git with version control 10. **Document with annotations** - Add context for other developers ## Troubleshooting **Pods not starting:** - Check image pull errors: `kubectl describe pod <pod-name>` - Verify resource availability: `kubectl get nodes` - Check events: `kubectl get events --sort-by='.lastTimestamp'` **Service not accessible:** - Verify selector matches pod labels: `kubectl get endpoints <service-name>` - Check service type and port configuration - Test from within cluster: `kubectl run debug --rm -it --image=busybox -- sh` **ConfigMap/Secret not loading:** - Verify names match in Deployment - Check namespace - Ensure resources exist: `kubectl get configmap,secret` ## Next Steps After creating manifests: 1. Store in Git repository 2. Set up CI/CD pipeline for deployment 3. Consider using Helm or Kustomize for templating 4. Implement GitOps with ArgoCD or Flux 5. Add monitoring and observability ## Related Skills - `helm-chart-scaffolding` - For templating and packaging - `gitops-workflow` - For automated deployments - `k8s-security-policies` - For advanced security configurations
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

k8s-security-policies

Implement Kubernetes security policies including NetworkPolicy,

architecture
⭐1
# Kubernetes Security Policies Comprehensive guide for implementing NetworkPolicy, PodSecurityPolicy, RBAC, and Pod Security Standards in Kubernetes. ## Purpose Implement defense-in-depth security for Kubernetes clusters using network policies, pod security standards, and RBAC. ## When to Use This Skill - Implement network segmentation - Configure pod security standards - Set up RBAC for least-privilege access - Create security policies for compliance - Implement admission control - Secure multi-tenant clusters ## Pod Security Standards ### 1. Privileged (Unrestricted) ```yaml apiVersion: v1 kind: Namespace metadata: name: privileged-ns labels: pod-security.kubernetes.io/enforce: privileged pod-security.kubernetes.io/audit: privileged pod-security.kubernetes.io/warn: privileged ``` ### 2. Baseline (Minimally restrictive) ```yaml apiVersion: v1 kind: Namespace metadata: name: baseline-ns labels: pod-security.kubernetes.io/enforce: baseline pod-security.kubernetes.io/audit: baseline pod-security.kubernetes.io/warn: baseline ``` ### 3. Restricted (Most restrictive) ```yaml apiVersion: v1 kind: Namespace metadata: name: restricted-ns labels: pod-security.kubernetes.io/enforce: restricted pod-security.kubernetes.io/audit: restricted pod-security.kubernetes.io/warn: restricted ``` ## Network Policies ### Default Deny All ```yaml apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: default-deny-all namespace: production spec: podSelector: {} policyTypes: - Ingress - Egress ``` ### Allow Frontend to Backend ```yaml apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: allow-frontend-to-backend namespace: production spec: podSelector: matchLabels: app: backend policyTypes: - Ingress ingress: - from: - podSelector: matchLabels: app: frontend ports: - protocol: TCP port: 8080 ``` ### Allow DNS ```yaml apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: allow-dns namespace: production spec: podSelector: {} policyTypes: - Egress egress: - to: - namespaceSelector: matchLabels: name: kube-system ports: - protocol: UDP port: 53 ``` **Reference:** See `assets/network-policy-template.yaml` ## RBAC Configuration ### Role (Namespace-scoped) ```yaml apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: pod-reader namespace: production rules: - apiGroups: [""] resources: ["pods"] verbs: ["get", "watch", "list"] ``` ### ClusterRole (Cluster-wide) ```yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: secret-reader rules: - apiGroups: [""] resources: ["secrets"] verbs: ["get", "watch", "list"] ``` ### RoleBinding ```yaml apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: read-pods namespace: production subjects: - kind: User name: jane apiGroup: rbac.authorization.k8s.io - kind: ServiceAccount name: default namespace: production roleRef: kind: Role name: pod-reader apiGroup: rbac.authorization.k8s.io ``` **Reference:** See `references/rbac-patterns.md` ## Pod Security Context ### Restricted Pod ```yaml apiVersion: v1 kind: Pod metadata: name: secure-pod spec: securityContext: runAsNonRoot: true runAsUser: 1000 fsGroup: 1000 seccompProfile: type: RuntimeDefault containers: - name: app image: myapp:1.0 securityContext: allowPrivilegeEscalation: false readOnlyRootFilesystem: true capabilities: drop: - ALL ``` ## Policy Enforcement with OPA Gatekeeper ### ConstraintTemplate ```yaml apiVersion: templates.gatekeeper.sh/v1 kind: ConstraintTemplate metadata: name: k8srequiredlabels spec: crd: spec: names: kind: K8sRequiredLabels validation: openAPIV3Schema: type: object properties: labels: type: array items: type: string targets: - target: admission.k8s.gatekeeper.sh rego: | package k8srequiredlabels violation[{"msg": msg, "details": {"missing_labels": missing}}] { provided := {label | input.review.object.metadata.labels[label]} required := {label | label := input.parameters.labels[_]} missing := required - provided count(missing) > 0 msg := sprintf("missing required labels: %v", [missing]) } ``` ### Constraint ```yaml apiVersion: constraints.gatekeeper.sh/v1beta1 kind: K8sRequiredLabels metadata: name: require-app-label spec: match: kinds: - apiGroups: ["apps"] kinds: ["Deployment"] parameters: labels: ["app", "environment"] ``` ## Service Mesh Security (Istio) ### PeerAuthentication (mTLS) ```yaml apiVersion: security.istio.io/v1beta1 kind: PeerAuthentication metadata: name: default namespace: production spec: mtls: mode: STRICT ``` ### AuthorizationPolicy ```yaml apiVersion: security.istio.io/v1beta1 kind: AuthorizationPolicy metadata: name: allow-frontend namespace: production spec: selector: matchLabels: app: backend action: ALLOW rules: - from: - source: principals: ["cluster.local/ns/production/sa/frontend"] ``` ## Best Practices 1. **Implement Pod Security Standards** at namespace level 2. **Use Network Policies** for network segmentation 3. **Apply least-privilege RBAC** for all service accounts 4. **Enable admission control** (OPA Gatekeeper/Kyverno) 5. **Run containers as non-root** 6. **Use read-only root filesystem** 7. **Drop all capabilities** unless needed 8. **Implement resource quotas** and limit ranges 9. **Enable audit logging** for security events 10. **Regular security scanning** of images ## Compliance Frameworks ### CIS Kubernetes Benchmark - Use RBAC authorization - Enable audit logging - Use Pod Security Standards - Configure network policies - Implement secrets encryption at rest - Enable node authentication ### NIST Cybersecurity Framework - Implement defense in depth - Use network segmentation - Configure security monitoring - Implement access controls - Enable logging and monitoring ## Troubleshooting **NetworkPolicy not working:** ```bash # Check if CNI supports NetworkPolicy kubectl get nodes -o wide kubectl describe networkpolicy <name> ``` **RBAC permission denied:** ```bash # Check effective permissions kubectl auth can-i list pods --as system:serviceaccount:default:my-sa kubectl auth can-i '*' '*' --as system:serviceaccount:default:my-sa ``` ## Reference Files - `assets/network-policy-template.yaml` - Network policy examples - `assets/pod-security-template.yaml` - Pod security policies - `references/rbac-patterns.md` - RBAC configuration patterns ## Related Skills - `k8s-manifest-generator` - For creating secure manifests - `gitops-workflow` - For automated policy deployment
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

prompt-engineering-patterns

Master advanced prompt engineering techniques to maximize LLM

coding
⭐1
# Prompt Engineering Patterns Master advanced prompt engineering techniques to maximize LLM performance, reliability, and controllability. ## When to Use This Skill - Designing complex prompts for production LLM applications - Optimizing prompt performance and consistency - Implementing structured reasoning patterns (chain-of-thought, tree-of-thought) - Building few-shot learning systems with dynamic example selection - Creating reusable prompt templates with variable interpolation - Debugging and refining prompts that produce inconsistent outputs - Implementing system prompts for specialized AI assistants - Using structured outputs (JSON mode) for reliable parsing ## Core Capabilities ### 1. Few-Shot Learning - Example selection strategies (semantic similarity, diversity sampling) - Balancing example count with context window constraints - Constructing effective demonstrations with input-output pairs - Dynamic example retrieval from knowledge bases - Handling edge cases through strategic example selection ### 2. Chain-of-Thought Prompting - Step-by-step reasoning elicitation - Zero-shot CoT with "Let's think step by step" - Few-shot CoT with reasoning traces - Self-consistency techniques (sampling multiple reasoning paths) - Verification and validation steps ### 3. Structured Outputs - JSON mode for reliable parsing - Pydantic schema enforcement - Type-safe response handling - Error handling for malformed outputs ### 4. Prompt Optimization - Iterative refinement workflows - A/B testing prompt variations - Measuring prompt performance metrics (accuracy, consistency, latency) - Reducing token usage while maintaining quality - Handling edge cases and failure modes ### 5. Template Systems - Variable interpolation and formatting - Conditional prompt sections - Multi-turn conversation templates - Role-based prompt composition - Modular prompt components ### 6. System Prompt Design - Setting model behavior and constraints - Defining output formats and structure - Establishing role and expertise - Safety guidelines and content policies - Context setting and background information ## Quick Start ```python from langchain_anthropic import ChatAnthropic from langchain_core.prompts import ChatPromptTemplate from pydantic import BaseModel, Field # Define structured output schema class SQLQuery(BaseModel): query: str = Field(description="The SQL query") explanation: str = Field(description="Brief explanation of what the query does") tables_used: list[str] = Field(description="List of tables referenced") # Initialize model with structured output llm = ChatAnthropic(model="claude-sonnet-4-6") structured_llm = llm.with_structured_output(SQLQuery) # Create prompt template prompt = ChatPromptTemplate.from_messages([ ("system", """You are an expert SQL developer. Generate efficient, secure SQL queries. Always use parameterized queries to prevent SQL injection. Explain your reasoning briefly."""), ("user", "Convert this to SQL: {query}") ]) # Create chain chain = prompt | structured_llm # Use result = await chain.ainvoke({ "query": "Find all users who registered in the last 30 days" }) print(result.query) print(result.explanation) ``` ## Key Patterns ### Pattern 1: Structured Output with Pydantic ```python from anthropic import Anthropic from pydantic import BaseModel, Field from typing import Literal import json class SentimentAnalysis(BaseModel): sentiment: Literal["positive", "negative", "neutral"] confidence: float = Field(ge=0, le=1) key_phrases: list[str] reasoning: str async def analyze_sentiment(text: str) -> SentimentAnalysis: """Analyze sentiment with structured output.""" client = Anthropic() message = client.messages.create( model="claude-sonnet-4-6", max_tokens=500, messages=[{ "role": "user", "content": f"""Analyze the sentiment of this text. Text: {text} Respond with JSON matching this schema: {{ "sentiment": "positive" | "negative" | "neutral", "confidence": 0.0-1.0, "key_phrases": ["phrase1", "phrase2"], "reasoning": "brief explanation" }}""" }] ) return SentimentAnalysis(**json.loads(message.content[0].text)) ``` ### Pattern 2: Chain-of-Thought with Self-Verification ```python from langchain_core.prompts import ChatPromptTemplate cot_prompt = ChatPromptTemplate.from_template(""" Solve this problem step by step. Problem: {problem} Instructions: 1. Break down the problem into clear steps 2. Work through each step showing your reasoning 3. State your final answer 4. Verify your answer by checking it against the original problem Format your response as: ## Steps [Your step-by-step reasoning] ## Answer [Your final answer] ## Verification [Check that your answer is correct] """) ``` ### Pattern 3: Few-Shot with Dynamic Example Selection ```python from langchain_voyageai import VoyageAIEmbeddings from langchain_core.example_selectors import SemanticSimilarityExampleSelector from langchain_chroma import Chroma # Create example selector with semantic similarity example_selector = SemanticSimilarityExampleSelector.from_examples( examples=[ {"input": "How do I reset my password?", "output": "Go to Settings > Security > Reset Password"}, {"input": "Where can I see my order history?", "output": "Navigate to Account > Orders"}, {"input": "How do I contact support?", "output": "Click Help > Contact Us or email support@example.com"}, ], embeddings=VoyageAIEmbeddings(model="voyage-3-large"), vectorstore_cls=Chroma, k=2 # Select 2 most similar examples ) async def get_few_shot_prompt(query: str) -> str: """Build prompt with dynamically selected examples.""" examples = await example_selector.aselect_examples({"input": query}) examples_text = "\n".join( f"User: {ex['input']}\nAssistant: {ex['output']}" for ex in examples ) return f"""You are a helpful customer support assistant. Here are some example interactions: {examples_text} Now respond to this query: User: {query} Assistant:""" ``` ### Pattern 4: Progressive Disclosure Start with simple prompts, add complexity only when needed: ```python PROMPT_LEVELS = { # Level 1: Direct instruction "simple": "Summarize this article: {text}", # Level 2: Add constraints "constrained": """Summarize this article in 3 bullet points, focusing on: - Key findings - Main conclusions - Practical implications Article: {text}""", # Level 3: Add reasoning "reasoning": """Read this article carefully. 1. First, identify the main topic and thesis 2. Then, extract the key supporting points 3. Finally, summarize in 3 bullet points Article: {text} Summary:""", # Level 4: Add examples "few_shot": """Read articles and provide concise summaries. Example: Article: "New research shows that regular exercise can reduce anxiety by up to 40%..." Summary: β€’ Regular exercise reduces anxiety by up to 40% β€’ 30 minutes of moderate activity 3x/week is sufficient β€’ Benefits appear within 2 weeks of starting Now summarize this article: Article: {text} Summary:""" } ``` ### Pattern 5: Error Recovery and Fallback ```python from pydantic import BaseModel, ValidationError import json class ResponseWithConfidence(BaseModel): answer: str confidence: float sources: list[str] alternative_interpretations: list[str] = [] ERROR_RECOVERY_PROMPT = """ Answer the question based on the context provided. Context: {context} Question: {question} Instructions: 1. If you can answer confidently (>0.8), provide a direct answer 2. If you're somewhat confident (0.5-0.8), provide your best answer with caveats 3. If you're uncertain (<0.5), explain what information is missing 4. Always provide alternative interpretations if the question is ambiguous Respond in JSON: {{ "answer": "your answer or 'I cannot determine this from the context'", "confidence": 0.0-1.0, "sources": ["relevant context excerpts"], "alternative_interpretations": ["if question is ambiguous"] }} """ async def answer_with_fallback( context: str, question: str, llm ) -> ResponseWithConfidence: """Answer with error recovery and fallback.""" prompt = ERROR_RECOVERY_PROMPT.format(context=context, question=question) try: response = await llm.ainvoke(prompt) return ResponseWithConfidence(**json.loads(response.content)) except (json.JSONDecodeError, ValidationError) as e: # Fallback: try to extract answer without structure simple_prompt = f"Based on: {context}\n\nAnswer: {question}" simple_response = await llm.ainvoke(simple_prompt) return ResponseWithConfidence( answer=simple_response.content, confidence=0.5, sources=["fallback extraction"], alternative_interpretations=[] ) ``` ### Pattern 6: Role-Based System Prompts ```python SYSTEM_PROMPTS = { "analyst": """You are a senior data analyst with expertise in SQL, Python, and business intelligence. Your responsibilities: - Write efficient, well-documented queries - Explain your analysis methodology - Highlight key insights and recommendations - Flag any data quality concerns Communication style: - Be precise and technical when discussing methodology - Translate technical findings into business impact - Use clear visualizations when helpful""", "assistant": """You are a helpful AI assistant focused on accuracy and clarity. Core principles: - Always cite sources when making factual claims - Acknowledge uncertainty rather than guessing - Ask clarifying questions when the request is ambiguous - Provide step-by-step explanations for complex topics Constraints: - Do not provide medical, legal, or financial advice - Redirect harmful requests appropriately - Protect user privacy""", "code_reviewer": """You are a senior software engineer conducting code reviews. Review criteria: - Correctness: Does the code work as intended? - Security: Are there any vulnerabilities? - Performance: Are there efficiency concerns? - Maintainability: Is the code readable and well-structured? - Best practices: Does it follow language idioms? Output format: 1. Summary assessment (approve/request changes) 2. Critical issues (must fix) 3. Suggestions (nice to have) 4. Positive feedback (what's done well)""" } ``` ## Integration Patterns ### With RAG Systems ```python RAG_PROMPT = """You are a knowledgeable assistant that answers questions based on provided context. Context (retrieved from knowledge base): {context} Instructions: 1. Answer ONLY based on the provided context 2. If the context doesn't contain the answer, say "I don't have information about that in my knowledge base" 3. Cite specific passages using [1], [2] notation 4. If the question is ambiguous, ask for clarification Question: {question} Answer:""" ``` ### With Validation and Verification ```python VALIDATED_PROMPT = """Complete the following task: Task: {task} After generating your response, verify it meets ALL these criteria: βœ“ Directly addresses the original request βœ“ Contains no factual errors βœ“ Is appropriately detailed (not too brief, not too verbose) βœ“ Uses proper formatting βœ“ Is safe and appropriate If verification fails on any criterion, revise before responding. Response:""" ``` ## Performance Optimization ### Token Efficiency ```python # Before: Verbose prompt (150+ tokens) verbose_prompt = """ I would like you to please take the following text and provide me with a comprehensive summary of the main points. The summary should capture the key ideas and important details while being concise and easy to understand. """ # After: Concise prompt (30 tokens) concise_prompt = """Summarize the key points concisely: {text} Summary:""" ``` ### Caching Common Prefixes ```python from anthropic import Anthropic client = Anthropic() # Use prompt caching for repeated system prompts response = client.messages.create( model="claude-sonnet-4-6", max_tokens=1000, system=[ { "type": "text", "text": LONG_SYSTEM_PROMPT, "cache_control": {"type": "ephemeral"} } ], messages=[{"role": "user", "content": user_query}] ) ``` ## Best Practices 1. **Be Specific**: Vague prompts produce inconsistent results 2. **Show, Don't Tell**: Examples are more effective than descriptions 3. **Use Structured Outputs**: Enforce schemas with Pydantic for reliability 4. **Test Extensively**: Evaluate on diverse, representative inputs 5. **Iterate Rapidly**: Small changes can have large impacts 6. **Monitor Performance**: Track metrics in production 7. **Version Control**: Treat prompts as code with proper versioning 8. **Document Intent**: Explain why prompts are structured as they are ## Common Pitfalls - **Over-engineering**: Starting with complex prompts before trying simple ones - **Example pollution**: Using examples that don't match the target task - **Context overflow**: Exceeding token limits with excessive examples - **Ambiguous instructions**: Leaving room for multiple interpretations - **Ignoring edge cases**: Not testing on unusual or boundary inputs - **No error handling**: Assuming outputs will always be well-formed - **Hardcoded values**: Not parameterizing prompts for reuse ## Success Metrics Track these KPIs for your prompts: - **Accuracy**: Correctness of outputs - **Consistency**: Reproducibility across similar inputs - **Latency**: Response time (P50, P95, P99) - **Token Usage**: Average tokens per request - **Success Rate**: Percentage of valid, parseable outputs - **User Satisfaction**: Ratings and feedback ## Resources - [Anthropic Prompt Engineering Guide](https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering) - [Claude Prompt Caching](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching) - [OpenAI Prompt Engineering](https://platform.openai.com/docs/guides/prompt-engineering) - [LangChain Prompts](https://python.langchain.com/docs/concepts/prompts/)
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

slo-implementation

Define and implement Service Level Indicators (SLIs) and Service

coding
⭐1
# SLO Implementation Framework for defining and implementing Service Level Indicators (SLIs), Service Level Objectives (SLOs), and error budgets. ## Purpose Implement measurable reliability targets using SLIs, SLOs, and error budgets to balance reliability with innovation velocity. ## When to Use - Define service reliability targets - Measure user-perceived reliability - Implement error budgets - Create SLO-based alerts - Track reliability goals ## SLI/SLO/SLA Hierarchy ``` SLA (Service Level Agreement) ↓ Contract with customers SLO (Service Level Objective) ↓ Internal reliability target SLI (Service Level Indicator) ↓ Actual measurement ``` ## Defining SLIs ### Common SLI Types #### 1. Availability SLI ```promql # Successful requests / Total requests sum(rate(http_requests_total{status!~"5.."}[28d])) / sum(rate(http_requests_total[28d])) ``` #### 2. Latency SLI ```promql # Requests below latency threshold / Total requests sum(rate(http_request_duration_seconds_bucket{le="0.5"}[28d])) / sum(rate(http_request_duration_seconds_count[28d])) ``` #### 3. Durability SLI ``` # Successful writes / Total writes sum(storage_writes_successful_total) / sum(storage_writes_total) ``` **Reference:** See `references/slo-definitions.md` ## Setting SLO Targets ### Availability SLO Examples | SLO % | Downtime/Month | Downtime/Year | | ------ | -------------- | ------------- | | 99% | 7.2 hours | 3.65 days | | 99.9% | 43.2 minutes | 8.76 hours | | 99.95% | 21.6 minutes | 4.38 hours | | 99.99% | 4.32 minutes | 52.56 minutes | ### Choose Appropriate SLOs **Consider:** - User expectations - Business requirements - Current performance - Cost of reliability - Competitor benchmarks **Example SLOs:** ```yaml slos: - name: api_availability target: 99.9 window: 28d sli: | sum(rate(http_requests_total{status!~"5.."}[28d])) / sum(rate(http_requests_total[28d])) - name: api_latency_p95 target: 99 window: 28d sli: | sum(rate(http_request_duration_seconds_bucket{le="0.5"}[28d])) / sum(rate(http_request_duration_seconds_count[28d])) ``` ## Error Budget Calculation ### Error Budget Formula ``` Error Budget = 1 - SLO Target ``` **Example:** - SLO: 99.9% availability - Error Budget: 0.1% = 43.2 minutes/month - Current Error: 0.05% = 21.6 minutes/month - Remaining Budget: 50% ### Error Budget Policy ```yaml error_budget_policy: - remaining_budget: 100% action: Normal development velocity - remaining_budget: 50% action: Consider postponing risky changes - remaining_budget: 10% action: Freeze non-critical changes - remaining_budget: 0% action: Feature freeze, focus on reliability ``` **Reference:** See `references/error-budget.md` ## SLO Implementation ### Prometheus Recording Rules ```yaml # SLI Recording Rules groups: - name: sli_rules interval: 30s rules: # Availability SLI - record: sli:http_availability:ratio expr: | sum(rate(http_requests_total{status!~"5.."}[28d])) / sum(rate(http_requests_total[28d])) # Latency SLI (requests < 500ms) - record: sli:http_latency:ratio expr: | sum(rate(http_request_duration_seconds_bucket{le="0.5"}[28d])) / sum(rate(http_request_duration_seconds_count[28d])) - name: slo_rules interval: 5m rules: # SLO compliance (1 = meeting SLO, 0 = violating) - record: slo:http_availability:compliance expr: sli:http_availability:ratio >= bool 0.999 - record: slo:http_latency:compliance expr: sli:http_latency:ratio >= bool 0.99 # Error budget remaining (percentage) - record: slo:http_availability:error_budget_remaining expr: | (sli:http_availability:ratio - 0.999) / (1 - 0.999) * 100 # Error budget burn rate - record: slo:http_availability:burn_rate_5m expr: | (1 - ( sum(rate(http_requests_total{status!~"5.."}[5m])) / sum(rate(http_requests_total[5m])) )) / (1 - 0.999) ``` ### SLO Alerting Rules ```yaml groups: - name: slo_alerts interval: 1m rules: # Fast burn: 14.4x rate, 1 hour window # Consumes 2% error budget in 1 hour - alert: SLOErrorBudgetBurnFast expr: | slo:http_availability:burn_rate_1h > 14.4 and slo:http_availability:burn_rate_5m > 14.4 for: 2m labels: severity: critical annotations: summary: "Fast error budget burn detected" description: "Error budget burning at {{ $value }}x rate" # Slow burn: 6x rate, 6 hour window # Consumes 5% error budget in 6 hours - alert: SLOErrorBudgetBurnSlow expr: | slo:http_availability:burn_rate_6h > 6 and slo:http_availability:burn_rate_30m > 6 for: 15m labels: severity: warning annotations: summary: "Slow error budget burn detected" description: "Error budget burning at {{ $value }}x rate" # Error budget exhausted - alert: SLOErrorBudgetExhausted expr: slo:http_availability:error_budget_remaining < 0 for: 5m labels: severity: critical annotations: summary: "SLO error budget exhausted" description: "Error budget remaining: {{ $value }}%" ``` ## SLO Dashboard **Grafana Dashboard Structure:** ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ SLO Compliance (Current) β”‚ β”‚ βœ“ 99.95% (Target: 99.9%) β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ Error Budget Remaining: 65% β”‚ β”‚ β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘ 65% β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ SLI Trend (28 days) β”‚ β”‚ [Time series graph] β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ Burn Rate Analysis β”‚ β”‚ [Burn rate by time window] β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` **Example Queries:** ```promql # Current SLO compliance sli:http_availability:ratio * 100 # Error budget remaining slo:http_availability:error_budget_remaining # Days until error budget exhausted (at current burn rate) (slo:http_availability:error_budget_remaining / 100) * 28 / (1 - sli:http_availability:ratio) * (1 - 0.999) ``` ## Multi-Window Burn Rate Alerts ```yaml # Combination of short and long windows reduces false positives rules: - alert: SLOBurnRateHigh expr: | ( slo:http_availability:burn_rate_1h > 14.4 and slo:http_availability:burn_rate_5m > 14.4 ) or ( slo:http_availability:burn_rate_6h > 6 and slo:http_availability:burn_rate_30m > 6 ) labels: severity: critical ``` ## SLO Review Process ### Weekly Review - Current SLO compliance - Error budget status - Trend analysis - Incident impact ### Monthly Review - SLO achievement - Error budget usage - Incident postmortems - SLO adjustments ### Quarterly Review - SLO relevance - Target adjustments - Process improvements - Tooling enhancements ## Best Practices 1. **Start with user-facing services** 2. **Use multiple SLIs** (availability, latency, etc.) 3. **Set achievable SLOs** (don't aim for 100%) 4. **Implement multi-window alerts** to reduce noise 5. **Track error budget** consistently 6. **Review SLOs regularly** 7. **Document SLO decisions** 8. **Align with business goals** 9. **Automate SLO reporting** 10. **Use SLOs for prioritization** ## Reference Files - `assets/slo-template.md` - SLO definition template - `references/slo-definitions.md` - SLO definition patterns - `references/error-budget.md` - Error budget calculations ## Related Skills - `prometheus-configuration` - For metric collection - `grafana-dashboards` - For SLO visualization
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

pci-compliance

Implement PCI DSS compliance requirements for secure handling of

business
⭐1
# PCI Compliance Master PCI DSS (Payment Card Industry Data Security Standard) compliance for secure payment processing and handling of cardholder data. ## When to Use This Skill - Building payment processing systems - Handling credit card information - Implementing secure payment flows - Conducting PCI compliance audits - Reducing PCI compliance scope - Implementing tokenization and encryption - Preparing for PCI DSS assessments ## PCI DSS Requirements (12 Core Requirements) ### Build and Maintain Secure Network 1. Install and maintain firewall configuration 2. Don't use vendor-supplied defaults for passwords ### Protect Cardholder Data 3. Protect stored cardholder data 4. Encrypt transmission of cardholder data across public networks ### Maintain Vulnerability Management 5. Protect systems against malware 6. Develop and maintain secure systems and applications ### Implement Strong Access Control 7. Restrict access to cardholder data by business need-to-know 8. Identify and authenticate access to system components 9. Restrict physical access to cardholder data ### Monitor and Test Networks 10. Track and monitor all access to network resources and cardholder data 11. Regularly test security systems and processes ### Maintain Information Security Policy 12. Maintain a policy that addresses information security ## Compliance Levels **Level 1**: > 6 million transactions/year (annual ROC required) **Level 2**: 1-6 million transactions/year (annual SAQ) **Level 3**: 20,000-1 million e-commerce transactions/year **Level 4**: < 20,000 e-commerce or < 1 million total transactions ## Data Minimization (Never Store) ```python # NEVER STORE THESE PROHIBITED_DATA = { 'full_track_data': 'Magnetic stripe data', 'cvv': 'Card verification code/value', 'pin': 'PIN or PIN block' } # CAN STORE (if encrypted) ALLOWED_DATA = { 'pan': 'Primary Account Number (card number)', 'cardholder_name': 'Name on card', 'expiration_date': 'Card expiration', 'service_code': 'Service code' } class PaymentData: """Safe payment data handling.""" def __init__(self): self.prohibited_fields = ['cvv', 'cvv2', 'cvc', 'pin'] def sanitize_log(self, data): """Remove sensitive data from logs.""" sanitized = data.copy() # Mask PAN if 'card_number' in sanitized: card = sanitized['card_number'] sanitized['card_number'] = f"{card[:6]}{'*' * (len(card) - 10)}{card[-4:]}" # Remove prohibited data for field in self.prohibited_fields: sanitized.pop(field, None) return sanitized def validate_no_prohibited_storage(self, data): """Ensure no prohibited data is being stored.""" for field in self.prohibited_fields: if field in data: raise SecurityError(f"Attempting to store prohibited field: {field}") ``` ## Tokenization ### Using Payment Processor Tokens ```python import stripe class TokenizedPayment: """Handle payments using tokens (no card data on server).""" @staticmethod def create_payment_method_token(card_details): """Create token from card details (client-side only).""" # THIS SHOULD ONLY BE DONE CLIENT-SIDE WITH STRIPE.JS # NEVER send card details to your server """ // Frontend JavaScript const stripe = Stripe('pk_...'); const {token, error} = await stripe.createToken({ card: { number: '4242424242424242', exp_month: 12, exp_year: 2024, cvc: '123' } }); // Send token.id to server (NOT card details) """ pass @staticmethod def charge_with_token(token_id, amount): """Charge using token (server-side).""" # Your server only sees the token, never the card number stripe.api_key = "sk_..." charge = stripe.Charge.create( amount=amount, currency="usd", source=token_id, # Token instead of card details description="Payment" ) return charge @staticmethod def store_payment_method(customer_id, payment_method_token): """Store payment method as token for future use.""" stripe.Customer.modify( customer_id, source=payment_method_token ) # Store only customer_id and payment_method_id in your database # NEVER store actual card details return { 'customer_id': customer_id, 'has_payment_method': True # DO NOT store: card number, CVV, etc. } ``` ### Custom Tokenization (Advanced) ```python import secrets from cryptography.fernet import Fernet class TokenVault: """Secure token vault for card data (if you must store it).""" def __init__(self, encryption_key): self.cipher = Fernet(encryption_key) self.vault = {} # In production: use encrypted database def tokenize(self, card_data): """Convert card data to token.""" # Generate secure random token token = secrets.token_urlsafe(32) # Encrypt card data encrypted = self.cipher.encrypt(json.dumps(card_data).encode()) # Store token -> encrypted data mapping self.vault[token] = encrypted return token def detokenize(self, token): """Retrieve card data from token.""" encrypted = self.vault.get(token) if not encrypted: raise ValueError("Token not found") # Decrypt decrypted = self.cipher.decrypt(encrypted) return json.loads(decrypted.decode()) def delete_token(self, token): """Remove token from vault.""" self.vault.pop(token, None) ``` ## Encryption ### Data at Rest ```python from cryptography.hazmat.primitives.ciphers.aead import AESGCM import os class EncryptedStorage: """Encrypt data at rest using AES-256-GCM.""" def __init__(self, encryption_key): """Initialize with 256-bit key.""" self.key = encryption_key # Must be 32 bytes def encrypt(self, plaintext): """Encrypt data.""" # Generate random nonce nonce = os.urandom(12) # Encrypt aesgcm = AESGCM(self.key) ciphertext = aesgcm.encrypt(nonce, plaintext.encode(), None) # Return nonce + ciphertext return nonce + ciphertext def decrypt(self, encrypted_data): """Decrypt data.""" # Extract nonce and ciphertext nonce = encrypted_data[:12] ciphertext = encrypted_data[12:] # Decrypt aesgcm = AESGCM(self.key) plaintext = aesgcm.decrypt(nonce, ciphertext, None) return plaintext.decode() # Usage storage = EncryptedStorage(os.urandom(32)) encrypted_pan = storage.encrypt("4242424242424242") # Store encrypted_pan in database ``` ### Data in Transit ```python # Always use TLS 1.2 or higher # Flask/Django example app.config['SESSION_COOKIE_SECURE'] = True # HTTPS only app.config['SESSION_COOKIE_HTTPONLY'] = True app.config['SESSION_COOKIE_SAMESITE'] = 'Strict' # Enforce HTTPS from flask_talisman import Talisman Talisman(app, force_https=True) ``` ## Access Control ```python from functools import wraps from flask import session def require_pci_access(f): """Decorator to restrict access to cardholder data.""" @wraps(f) def decorated_function(*args, **kwargs): user = session.get('user') # Check if user has PCI access role if not user or 'pci_access' not in user.get('roles', []): return {'error': 'Unauthorized access to cardholder data'}, 403 # Log access attempt audit_log( user=user['id'], action='access_cardholder_data', resource=f.__name__ ) return f(*args, **kwargs) return decorated_function @app.route('/api/payment-methods') @require_pci_access def get_payment_methods(): """Retrieve payment methods (restricted access).""" # Only accessible to users with pci_access role pass ``` ## Audit Logging ```python import logging from datetime import datetime class PCIAuditLogger: """PCI-compliant audit logging.""" def __init__(self): self.logger = logging.getLogger('pci_audit') # Configure to write to secure, append-only log def log_access(self, user_id, resource, action, result): """Log access to cardholder data.""" entry = { 'timestamp': datetime.utcnow().isoformat(), 'user_id': user_id, 'resource': resource, 'action': action, 'result': result, 'ip_address': request.remote_addr } self.logger.info(json.dumps(entry)) def log_authentication(self, user_id, success, method): """Log authentication attempt.""" entry = { 'timestamp': datetime.utcnow().isoformat(), 'user_id': user_id, 'event': 'authentication', 'success': success, 'method': method, 'ip_address': request.remote_addr } self.logger.info(json.dumps(entry)) # Usage audit = PCIAuditLogger() audit.log_access(user_id=123, resource='payment_methods', action='read', result='success') ``` ## Security Best Practices ### Input Validation ```python import re def validate_card_number(card_number): """Validate card number format (Luhn algorithm).""" # Remove spaces and dashes card_number = re.sub(r'[\s-]', '', card_number) # Check if all digits if not card_number.isdigit(): return False # Luhn algorithm def luhn_checksum(card_num): def digits_of(n): return [int(d) for d in str(n)] digits = digits_of(card_num) odd_digits = digits[-1::-2] even_digits = digits[-2::-2] checksum = sum(odd_digits) for d in even_digits: checksum += sum(digits_of(d * 2)) return checksum % 10 return luhn_checksum(card_number) == 0 def sanitize_input(user_input): """Sanitize user input to prevent injection.""" # Remove special characters # Validate against expected format # Escape for database queries pass ``` ## PCI DSS SAQ (Self-Assessment Questionnaire) ### SAQ A (Least Requirements) - E-commerce using hosted payment page - No card data on your systems - ~20 questions ### SAQ A-EP - E-commerce with embedded payment form - Uses JavaScript to handle card data - ~180 questions ### SAQ D (Most Requirements) - Store, process, or transmit card data - Full PCI DSS requirements - ~300 questions ## Compliance Checklist ```python PCI_COMPLIANCE_CHECKLIST = { 'network_security': [ 'Firewall configured and maintained', 'No vendor default passwords', 'Network segmentation implemented' ], 'data_protection': [ 'No storage of CVV, track data, or PIN', 'PAN encrypted when stored', 'PAN masked when displayed', 'Encryption keys properly managed' ], 'vulnerability_management': [ 'Anti-virus installed and updated', 'Secure development practices', 'Regular security patches', 'Vulnerability scanning performed' ], 'access_control': [ 'Access restricted by role', 'Unique IDs for all users', 'Multi-factor authentication', 'Physical security measures' ], 'monitoring': [ 'Audit logs enabled', 'Log review process', 'File integrity monitoring', 'Regular security testing' ], 'policy': [ 'Security policy documented', 'Risk assessment performed', 'Security awareness training', 'Incident response plan' ] } ``` ## Resources - **references/data-minimization.md**: Never store prohibited data - **references/tokenization.md**: Tokenization strategies - **references/encryption.md**: Encryption requirements - **references/access-control.md**: Role-based access - **references/audit-logging.md**: Comprehensive logging - **assets/pci-compliance-checklist.md**: Complete checklist - **assets/encrypted-storage.py**: Encryption utilities - **scripts/audit-payment-system.sh**: Compliance audit script ## Common Violations 1. **Storing CVV**: Never store card verification codes 2. **Unencrypted PAN**: Card numbers must be encrypted at rest 3. **Weak Encryption**: Use AES-256 or equivalent 4. **No Access Controls**: Restrict who can access cardholder data 5. **Missing Audit Logs**: Must log all access to payment data 6. **Insecure Transmission**: Always use TLS 1.2+ 7. **Default Passwords**: Change all default credentials 8. **No Security Testing**: Regular penetration testing required ## Reducing PCI Scope 1. **Use Hosted Payments**: Stripe Checkout, PayPal, etc. 2. **Tokenization**: Replace card data with tokens 3. **Network Segmentation**: Isolate cardholder data environment 4. **Outsource**: Use PCI-compliant payment processors 5. **No Storage**: Never store full card details By minimizing systems that touch card data, you reduce compliance burden significantly.
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

sast-configuration

Configure Static Application Security Testing (SAST) tools for

security
⭐1
# SAST Configuration Static Application Security Testing (SAST) tool setup, configuration, and custom rule creation for comprehensive security scanning across multiple programming languages. ## Overview This skill provides comprehensive guidance for setting up and configuring SAST tools including Semgrep, SonarQube, and CodeQL. Use this skill when you need to: - Set up SAST scanning in CI/CD pipelines - Create custom security rules for your codebase - Configure quality gates and compliance policies - Optimize scan performance and reduce false positives - Integrate multiple SAST tools for defense-in-depth ## Core Capabilities ### 1. Semgrep Configuration - Custom rule creation with pattern matching - Language-specific security rules (Python, JavaScript, Go, Java, etc.) - CI/CD integration (GitHub Actions, GitLab CI, Jenkins) - False positive tuning and rule optimization - Organizational policy enforcement ### 2. SonarQube Setup - Quality gate configuration - Security hotspot analysis - Code coverage and technical debt tracking - Custom quality profiles for languages - Enterprise integration with LDAP/SAML ### 3. CodeQL Analysis - GitHub Advanced Security integration - Custom query development - Vulnerability variant analysis - Security research workflows - SARIF result processing ## Quick Start ### Initial Assessment 1. Identify primary programming languages in your codebase 2. Determine compliance requirements (PCI-DSS, SOC 2, etc.) 3. Choose SAST tool based on language support and integration needs 4. Review baseline scan to understand current security posture ### Basic Setup ```bash # Semgrep quick start pip install semgrep semgrep --config=auto --error # SonarQube with Docker docker run -d --name sonarqube -p 9000:9000 sonarqube:latest # CodeQL CLI setup gh extension install github/gh-codeql codeql database create mydb --language=python ``` ## Reference Documentation - [Semgrep Rule Creation](references/semgrep-rules.md) - Pattern-based security rule development - [SonarQube Configuration](references/sonarqube-config.md) - Quality gates and profiles - [CodeQL Setup Guide](references/codeql-setup.md) - Query development and workflows ## Templates & Assets - [semgrep-config.yml](assets/semgrep-config.yml) - Production-ready Semgrep configuration - [sonarqube-settings.xml](assets/sonarqube-settings.xml) - SonarQube quality profile template - [run-sast.sh](scripts/run-sast.sh) - Automated SAST execution script ## Integration Patterns ### CI/CD Pipeline Integration ```yaml # GitHub Actions example - name: Run Semgrep uses: returntocorp/semgrep-action@v1 with: config: >- p/security-audit p/owasp-top-ten ``` ### Pre-commit Hook ```bash # .pre-commit-config.yaml - repo: https://github.com/returntocorp/semgrep rev: v1.45.0 hooks: - id: semgrep args: ['--config=auto', '--error'] ``` ## Best Practices 1. **Start with Baseline** - Run initial scan to establish security baseline - Prioritize critical and high severity findings - Create remediation roadmap 2. **Incremental Adoption** - Begin with security-focused rules - Gradually add code quality rules - Implement blocking only for critical issues 3. **False Positive Management** - Document legitimate suppressions - Create allow lists for known safe patterns - Regularly review suppressed findings 4. **Performance Optimization** - Exclude test files and generated code - Use incremental scanning for large codebases - Cache scan results in CI/CD 5. **Team Enablement** - Provide security training for developers - Create internal documentation for common patterns - Establish security champions program ## Common Use Cases ### New Project Setup ```bash ./scripts/run-sast.sh --setup --language python --tools semgrep,sonarqube ``` ### Custom Rule Development ```yaml # See references/semgrep-rules.md for detailed examples rules: - id: hardcoded-jwt-secret pattern: jwt.encode($DATA, "...", ...) message: JWT secret should not be hardcoded severity: ERROR ``` ### Compliance Scanning ```bash # PCI-DSS focused scan semgrep --config p/pci-dss --json -o pci-scan-results.json ``` ## Troubleshooting ### High False Positive Rate - Review and tune rule sensitivity - Add path filters to exclude test files - Use nostmt metadata for noisy patterns - Create organization-specific rule exceptions ### Performance Issues - Enable incremental scanning - Parallelize scans across modules - Optimize rule patterns for efficiency - Cache dependencies and scan results ### Integration Failures - Verify API tokens and credentials - Check network connectivity and proxy settings - Review SARIF output format compatibility - Validate CI/CD runner permissions ## Related Skills - [OWASP Top 10 Checklist](../owasp-top10-checklist/SKILL.md) - [Container Security](../container-security/SKILL.md) - [Dependency Scanning](../dependency-scanning/SKILL.md) ## Tool Comparison | Tool | Best For | Language Support | Cost | Integration | | --------- | ------------------------ | ---------------- | --------------- | ------------- | | Semgrep | Custom rules, fast scans | 30+ languages | Free/Enterprise | Excellent | | SonarQube | Code quality + security | 25+ languages | Free/Commercial | Good | | CodeQL | Deep analysis, research | 10+ languages | Free (OSS) | GitHub native | ## Next Steps 1. Complete initial SAST tool setup 2. Run baseline security scan 3. Create custom rules for organization-specific patterns 4. Integrate into CI/CD pipeline 5. Establish security gate policies 6. Train development team on findings and remediation
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered