Skip to main content
EVOKORE// BROWSE
>

./browse/prompts

38 NODES
πŸ€–system promptβ€’6 months ago

Hook Observability and Session Replay

Instrument hooks with replayable logs, task-state visibility, and non-blocking observability around agent sessions.

devops
⭐1
# Hook Observability and Session Replay Imported from curated first-party documentation sources. ## What this covers Use this skill when you need to understand what hooks fired, what they emitted, and how a session can be replayed after the fact. ## Use this when - Debugging hook-driven automation - Replaying session events after failures - Adding observability without blocking interactive work ## Expected outcomes - Hook activity becomes inspectable and replayable - Operational state survives beyond a single terminal session - Observability stays useful without overwhelming operators ## Source synthesis - EVOKORE-MCP/docs/VOICE_AND_HOOKS.md (https://github.com/mattmre/EVOKORE-MCP/blob/main/docs/VOICE_AND_HOOKS.md) - EVOKORE-MCP/docs/USE_CASES_AND_WALKTHROUGHS.md (https://github.com/mattmre/EVOKORE-MCP/blob/main/docs/USE_CASES_AND_WALKTHROUGHS.md) ## Dedupe notes Focuses on replay and observability instead of importing the broader voice-sidecar guide verbatim. ## Source excerpts ### EVOKORE-MCP/docs/VOICE_AND_HOOKS.md EVOKORE currently has three separate voice-related systems plus a set of hook and observability utilities. They overlap in operator workflows, but they are not the same runtime. ## The three voice-related systems ### 1. ElevenLabs MCP proxy This is the optional `elevenlabs` child server configured in `mcp.config.json`. What it is: - proxied through the EVOKORE router - exposed as prefixed MCP tools - available to any EVOKORE-connected MCP client when configured successfully What it is for: - text-to-speech and other ElevenLabs MCP operations as tools - routing voice-related actions through the standard EVOKORE proxy/security stack Requirements: - `uvx` available on PATH - `ELEVENLABS_API_KEY` set ### 2. VoiceMode VoiceMode is a separate voice-conversation system for Claude Code. What it is: - registered separately from EVOKORE - not routed through EVOKOREÒ€ℒs stdio server - used for bidirectional voice conversation in Claude Code What it is for: - speaking to Claude and hearing spoken responses - using `OPENAI_API_KEY` and VoiceModeÒ€ℒs own runtime Windows note: - VoiceMode relies on `uvx` being directly available - set `OPENAI_API_KEY` in the shell that launches Claude Code ### 3. VoiceSidecar VoiceSidecar is a standalone WebSocket server implemented in `src/Voice ... ### EVOKORE-MCP/docs/USE_CASES_AND_WALKTHROUGHS.md This guide turns the runtime contracts into practical operator flows. ## Walkthrough 1: Adopt a workflow from the skill library Use this when you want EVOKORE to retrieve process guidance before the model starts acting. ### Goal Find and adopt an existing workflow such as `session-wrap`. ### Steps 1. Ask the client to search skills: ```text Search the MCP for a workflow about session wrap-up and continuity. ``` 2. EVOKORE uses `search_skills` and returns matching skills. 3. Ask for a specific skill: ```text Show me help for the session-wrap skill. ``` 4. EVOKORE uses `get_skill_help` and returns the skillÒ€ℒs internal instructions. 5. For broader task matching, ask: ```text Resolve a workflow for wrapping this session, documenting open risks, and preparing the next handoff. ``` 6. EVOKORE uses `resolve_workflow` and injects the top 1-3 relevant workflows directly into the tool response. ### Why this matters - keeps the model grounded in repo-specific process - reduces prompt drift - makes handoff and governance behavior repeatable ## Walkthrough 2: Use a proxied tool that requires HITL approval Use this when the tool is configured as `require_approval` in `permissions.yml`. ### Goal Allow a protected proxied tool call such as `fs_write_f ...
πŸ‘0
πŸ‘οΈ0
docs
πŸ“textβ€’6 months ago

Improvement Cycle Review Wizard

Run a live improvement-cycle workflow that links review artifacts, approvals, tool requests, and operator checkpoints.

productivity
⭐1
# Improvement Cycle Review Wizard Imported from curated first-party documentation sources. ## What this covers Use this workflow when an operator needs a guided improvement cycle with live status, explicit approvals, and linked artifacts. ## Use this when - Operational review sessions with live progress - Approval-heavy improvement cycles - Coordinating review artifacts with execution steps ## Expected outcomes - Live workflow execution remains visible to operators - Artifacts, approvals, and tool requests stay connected - Improvement loops become easier to audit and rerun ## Source synthesis - AGENT33/docs/phase25-26-live-review-walkthrough.md (https://github.com/mattmre/AGENT33/blob/main/docs/phase25-26-live-review-walkthrough.md) - AGENT33/docs/operator-improvement-cycle-and-jupyter.md (https://github.com/mattmre/AGENT33/blob/main/docs/operator-improvement-cycle-and-jupyter.md) ## Dedupe notes Merges AGENT33 live review walkthrough details with the shorter operator-focused improvement-cycle guide. ## Source excerpts ### AGENT33/docs/phase25-26-live-review-walkthrough.md This guide documents the operator flow introduced by the Phase 25 live workflow transport and the Phase 26/27 improvement-cycle review wizard stack. ## Scope - Live workflow execution with run-scoped graph refresh - WebSocket-first status streaming with authenticated SSE fallback - Improvement-cycle preset creation - Explanation artifact generation for `plan_review` and `diff_review` - Linked review creation, risk assessment, L1/L2 signoff, and final approval - Pending tool-approval triage inside the same workflow domain > Note > This walkthrough reflects the review stack built on `codex/session58-phase26-wizard` and validated in `codex/session60-phase22-docs-validation`. If the related PRs are still open, `main` may not yet expose every surface described here. ## Operator Flow ```mermaid sequenceDiagram participant UI as "Frontend Control Plane" participant WF as "Workflow APIs" participant VIZ as "Visualization APIs" participant EXP as "Explanation APIs" participant REV as "Review APIs" participant HITL as "Tool Approval APIs" UI->>WF: "POST /v1/workflows/{name}/execute (single mode, caller run_id)" UI->>VIZ: "GET /v1/visualizations/workflows/{workflow_id}/graph?run_id=..." UI->>WF: "WS /v1/workflows/{run_id}/ws" alt "WebSocket unav ... ### AGENT33/docs/operator-improvement-cycle-and-jupyter.md This guide covers the merged operator surfaces for: - the Phase 26 improvement-cycle review wizard - the Phase 27 canonical workflow presets - the Phase 38 Docker-backed Jupyter kernel workflow Use it when you want the shortest current path from UI entry point to a real workflow run. ## 1. Improvement-Cycle Wizard The wizard is mounted inside the frontend control plane under the `Workflows` domain. ### Entry path 1. Open the frontend at `http://localhost:3000` 2. Authenticate with a bearer token or API key 3. Open `Advanced Settings` 4. Select the `Workflows` domain 5. Use the `Improvement Cycle Wizard` panel at the top of the page ### What the wizard does The wizard stitches together the backend surfaces that previously had to be called manually: - plan review / diff review generation - review creation and risk assessment - L1 and L2 review submission - tool approval request review and decision capture Reference implementation: - frontend: `frontend/src/features/improvement-cycle/ImprovementCycleWizard.tsx` - tests: `frontend/src/features/improvement-cycle/ImprovementCycleWizard.test.tsx` For the detailed review flow, keep using: - [`phase25-26-live-review-walkthrough.md`](phase25-26-live-review-walkthrough.md) ## 2. Canonical Workflow Presets The `Workflows` doma ...
πŸ‘0
πŸ‘οΈ0
docs
πŸ€–system promptβ€’7 months ago

workflow-orchestration-patterns

Design durable workflows with Temporal for distributed systems.

coding
⭐1
# Workflow Orchestration Patterns Master workflow orchestration architecture with Temporal, covering fundamental design decisions, resilience patterns, and best practices for building reliable distributed systems. ## When to Use Workflow Orchestration ### Ideal Use Cases (Source: docs.temporal.io) - **Multi-step processes** spanning machines/services/databases - **Distributed transactions** requiring all-or-nothing semantics - **Long-running workflows** (hours to years) with automatic state persistence - **Failure recovery** that must resume from last successful step - **Business processes**: bookings, orders, campaigns, approvals - **Entity lifecycle management**: inventory tracking, account management, cart workflows - **Infrastructure automation**: CI/CD pipelines, provisioning, deployments - **Human-in-the-loop** systems requiring timeouts and escalations ### When NOT to Use - Simple CRUD operations (use direct API calls) - Pure data processing pipelines (use Airflow, batch processing) - Stateless request/response (use standard APIs) - Real-time streaming (use Kafka, event processors) ## Critical Design Decision: Workflows vs Activities **The Fundamental Rule** (Source: temporal.io/blog/workflow-engine-principles): - **Workflows** = Orchestration logic and decision-making - **Activities** = External interactions (APIs, databases, network calls) ### Workflows (Orchestration) **Characteristics:** - Contain business logic and coordination - **MUST be deterministic** (same inputs β†’ same outputs) - **Cannot** perform direct external calls - State automatically preserved across failures - Can run for years despite infrastructure failures **Example workflow tasks:** - Decide which steps to execute - Handle compensation logic - Manage timeouts and retries - Coordinate child workflows ### Activities (External Interactions) **Characteristics:** - Handle all external system interactions - Can be non-deterministic (API calls, DB writes) - Include built-in timeouts and retry logic - **Must be idempotent** (calling N times = calling once) - Short-lived (seconds to minutes typically) **Example activity tasks:** - Call payment gateway API - Write to database - Send emails or notifications - Query external services ### Design Decision Framework ``` Does it touch external systems? β†’ Activity Is it orchestration/decision logic? β†’ Workflow ``` ## Core Workflow Patterns ### 1. Saga Pattern with Compensation **Purpose**: Implement distributed transactions with rollback capability **Pattern** (Source: temporal.io/blog/compensating-actions-part-of-a-complete-breakfast-with-sagas): ``` For each step: 1. Register compensation BEFORE executing 2. Execute the step (via activity) 3. On failure, run all compensations in reverse order (LIFO) ``` **Example: Payment Workflow** 1. Reserve inventory (compensation: release inventory) 2. Charge payment (compensation: refund payment) 3. Fulfill order (compensation: cancel fulfillment) **Critical Requirements:** - Compensations must be idempotent - Register compensation BEFORE executing step - Run compensations in reverse order - Handle partial failures gracefully ### 2. Entity Workflows (Actor Model) **Purpose**: Long-lived workflow representing single entity instance **Pattern** (Source: docs.temporal.io/evaluate/use-cases-design-patterns): - One workflow execution = one entity (cart, account, inventory item) - Workflow persists for entity lifetime - Receives signals for state changes - Supports queries for current state **Example Use Cases:** - Shopping cart (add items, checkout, expiration) - Bank account (deposits, withdrawals, balance checks) - Product inventory (stock updates, reservations) **Benefits:** - Encapsulates entity behavior - Guarantees consistency per entity - Natural event sourcing ### 3. Fan-Out/Fan-In (Parallel Execution) **Purpose**: Execute multiple tasks in parallel, aggregate results **Pattern:** - Spawn child workflows or parallel activities - Wait for all to complete - Aggregate results - Handle partial failures **Scaling Rule** (Source: temporal.io/blog/workflow-engine-principles): - Don't scale individual workflows - For 1M tasks: spawn 1K child workflows Γ— 1K tasks each - Keep each workflow bounded ### 4. Async Callback Pattern **Purpose**: Wait for external event or human approval **Pattern:** - Workflow sends request and waits for signal - External system processes asynchronously - Sends signal to resume workflow - Workflow continues with response **Use Cases:** - Human approval workflows - Webhook callbacks - Long-running external processes ## State Management and Determinism ### Automatic State Preservation **How Temporal Works** (Source: docs.temporal.io/workflows): - Complete program state preserved automatically - Event History records every command and event - Seamless recovery from crashes - Applications restore pre-failure state ### Determinism Constraints **Workflows Execute as State Machines**: - Replay behavior must be consistent - Same inputs β†’ identical outputs every time **Prohibited in Workflows** (Source: docs.temporal.io/workflows): - ❌ Threading, locks, synchronization primitives - ❌ Random number generation (`random()`) - ❌ Global state or static variables - ❌ System time (`datetime.now()`) - ❌ Direct file I/O or network calls - ❌ Non-deterministic libraries **Allowed in Workflows**: - βœ… `workflow.now()` (deterministic time) - βœ… `workflow.random()` (deterministic random) - βœ… Pure functions and calculations - βœ… Calling activities (non-deterministic operations) ### Versioning Strategies **Challenge**: Changing workflow code while old executions still running **Solutions**: 1. **Versioning API**: Use `workflow.get_version()` for safe changes 2. **New Workflow Type**: Create new workflow, route new executions to it 3. **Backward Compatibility**: Ensure old events replay correctly ## Resilience and Error Handling ### Retry Policies **Default Behavior**: Temporal retries activities forever **Configure Retry**: - Initial retry interval - Backoff coefficient (exponential backoff) - Maximum interval (cap retry delay) - Maximum attempts (eventually fail) **Non-Retryable Errors**: - Invalid input (validation failures) - Business rule violations - Permanent failures (resource not found) ### Idempotency Requirements **Why Critical** (Source: docs.temporal.io/activities): - Activities may execute multiple times - Network failures trigger retries - Duplicate execution must be safe **Implementation Strategies**: - Idempotency keys (deduplication) - Check-then-act with unique constraints - Upsert operations instead of insert - Track processed request IDs ### Activity Heartbeats **Purpose**: Detect stalled long-running activities **Pattern**: - Activity sends periodic heartbeat - Includes progress information - Timeout if no heartbeat received - Enables progress-based retry ## Best Practices ### Workflow Design 1. **Keep workflows focused** - Single responsibility per workflow 2. **Small workflows** - Use child workflows for scalability 3. **Clear boundaries** - Workflow orchestrates, activities execute 4. **Test locally** - Use time-skipping test environment ### Activity Design 1. **Idempotent operations** - Safe to retry 2. **Short-lived** - Seconds to minutes, not hours 3. **Timeout configuration** - Always set timeouts 4. **Heartbeat for long tasks** - Report progress 5. **Error handling** - Distinguish retryable vs non-retryable ### Common Pitfalls **Workflow Violations**: - Using `datetime.now()` instead of `workflow.now()` - Threading or async operations in workflow code - Calling external APIs directly from workflow - Non-deterministic logic in workflows **Activity Mistakes**: - Non-idempotent operations (can't handle retries) - Missing timeouts (activities run forever) - No error classification (retry validation errors) - Ignoring payload limits (2MB per argument) ### Operational Considerations **Monitoring**: - Workflow execution duration - Activity failure rates - Retry attempts and backoff - Pending workflow counts **Scalability**: - Horizontal scaling with workers - Task queue partitioning - Child workflow decomposition - Activity batching when appropriate ## Additional Resources **Official Documentation**: - Temporal Core Concepts: docs.temporal.io/workflows - Workflow Patterns: docs.temporal.io/evaluate/use-cases-design-patterns - Best Practices: docs.temporal.io/develop/best-practices - Saga Pattern: temporal.io/blog/saga-pattern-made-easy **Key Principles**: 1. Workflows = orchestration, Activities = external calls 2. Determinism is non-negotiable for workflows 3. Idempotency is critical for activities 4. State preservation is automatic 5. Design for failure and recovery
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

web3-testing

Test smart contracts comprehensively using Hardhat and Foundry with

coding
⭐1
# Web3 Smart Contract Testing Master comprehensive testing strategies for smart contracts using Hardhat, Foundry, and advanced testing patterns. ## When to Use This Skill - Writing unit tests for smart contracts - Setting up integration test suites - Performing gas optimization testing - Fuzzing for edge cases - Forking mainnet for realistic testing - Automating test coverage reporting - Verifying contracts on Etherscan ## Hardhat Testing Setup ```javascript // hardhat.config.js require("@nomicfoundation/hardhat-toolbox"); require("@nomiclabs/hardhat-etherscan"); require("hardhat-gas-reporter"); require("solidity-coverage"); module.exports = { solidity: { version: "0.8.19", settings: { optimizer: { enabled: true, runs: 200, }, }, }, networks: { hardhat: { forking: { url: process.env.MAINNET_RPC_URL, blockNumber: 15000000, }, }, goerli: { url: process.env.GOERLI_RPC_URL, accounts: [process.env.PRIVATE_KEY], }, }, gasReporter: { enabled: true, currency: "USD", coinmarketcap: process.env.COINMARKETCAP_API_KEY, }, etherscan: { apiKey: process.env.ETHERSCAN_API_KEY, }, }; ``` ## Unit Testing Patterns ```javascript const { expect } = require("chai"); const { ethers } = require("hardhat"); const { loadFixture, time, } = require("@nomicfoundation/hardhat-network-helpers"); describe("Token Contract", function () { // Fixture for test setup async function deployTokenFixture() { const [owner, addr1, addr2] = await ethers.getSigners(); const Token = await ethers.getContractFactory("Token"); const token = await Token.deploy(); return { token, owner, addr1, addr2 }; } describe("Deployment", function () { it("Should set the right owner", async function () { const { token, owner } = await loadFixture(deployTokenFixture); expect(await token.owner()).to.equal(owner.address); }); it("Should assign total supply to owner", async function () { const { token, owner } = await loadFixture(deployTokenFixture); const ownerBalance = await token.balanceOf(owner.address); expect(await token.totalSupply()).to.equal(ownerBalance); }); }); describe("Transactions", function () { it("Should transfer tokens between accounts", async function () { const { token, owner, addr1 } = await loadFixture(deployTokenFixture); await expect(token.transfer(addr1.address, 50)).to.changeTokenBalances( token, [owner, addr1], [-50, 50], ); }); it("Should fail if sender doesn't have enough tokens", async function () { const { token, addr1 } = await loadFixture(deployTokenFixture); const initialBalance = await token.balanceOf(addr1.address); await expect( token.connect(addr1).transfer(owner.address, 1), ).to.be.revertedWith("Insufficient balance"); }); it("Should emit Transfer event", async function () { const { token, owner, addr1 } = await loadFixture(deployTokenFixture); await expect(token.transfer(addr1.address, 50)) .to.emit(token, "Transfer") .withArgs(owner.address, addr1.address, 50); }); }); describe("Time-based tests", function () { it("Should handle time-locked operations", async function () { const { token } = await loadFixture(deployTokenFixture); // Increase time by 1 day await time.increase(86400); // Test time-dependent functionality }); }); describe("Gas optimization", function () { it("Should use gas efficiently", async function () { const { token } = await loadFixture(deployTokenFixture); const tx = await token.transfer(addr1.address, 100); const receipt = await tx.wait(); expect(receipt.gasUsed).to.be.lessThan(50000); }); }); }); ``` ## Foundry Testing (Forge) ```solidity // SPDX-License-Identifier: MIT pragma solidity ^0.8.0; import "forge-std/Test.sol"; import "../src/Token.sol"; contract TokenTest is Test { Token token; address owner = address(1); address user1 = address(2); address user2 = address(3); function setUp() public { vm.prank(owner); token = new Token(); } function testInitialSupply() public { assertEq(token.totalSupply(), 1000000 * 10**18); } function testTransfer() public { vm.prank(owner); token.transfer(user1, 100); assertEq(token.balanceOf(user1), 100); assertEq(token.balanceOf(owner), token.totalSupply() - 100); } function testFailTransferInsufficientBalance() public { vm.prank(user1); token.transfer(user2, 100); // Should fail } function testCannotTransferToZeroAddress() public { vm.prank(owner); vm.expectRevert("Invalid recipient"); token.transfer(address(0), 100); } // Fuzzing test function testFuzzTransfer(uint256 amount) public { vm.assume(amount > 0 && amount <= token.totalSupply()); vm.prank(owner); token.transfer(user1, amount); assertEq(token.balanceOf(user1), amount); } // Test with cheatcodes function testDealAndPrank() public { // Give ETH to address vm.deal(user1, 10 ether); // Impersonate address vm.prank(user1); // Test functionality assertEq(user1.balance, 10 ether); } // Mainnet fork test function testForkMainnet() public { vm.createSelectFork("https://eth-mainnet.alchemyapi.io/v2/..."); // Interact with mainnet contracts address dai = 0x6B175474E89094C44Da98b954EedeAC495271d0F; assertEq(IERC20(dai).symbol(), "DAI"); } } ``` ## Advanced Testing Patterns ### Snapshot and Revert ```javascript describe("Complex State Changes", function () { let snapshotId; beforeEach(async function () { snapshotId = await network.provider.send("evm_snapshot"); }); afterEach(async function () { await network.provider.send("evm_revert", [snapshotId]); }); it("Test 1", async function () { // Make state changes }); it("Test 2", async function () { // State reverted, clean slate }); }); ``` ### Mainnet Forking ```javascript describe("Mainnet Fork Tests", function () { let uniswapRouter, dai, usdc; before(async function () { await network.provider.request({ method: "hardhat_reset", params: [ { forking: { jsonRpcUrl: process.env.MAINNET_RPC_URL, blockNumber: 15000000, }, }, ], }); // Connect to existing mainnet contracts uniswapRouter = await ethers.getContractAt( "IUniswapV2Router", "0x7a250d5630B4cF539739dF2C5dAcb4c659F2488D", ); dai = await ethers.getContractAt( "IERC20", "0x6B175474E89094C44Da98b954EedeAC495271d0F", ); }); it("Should swap on Uniswap", async function () { // Test with real Uniswap contracts }); }); ``` ### Impersonating Accounts ```javascript it("Should impersonate whale account", async function () { const whaleAddress = "0x..."; await network.provider.request({ method: "hardhat_impersonateAccount", params: [whaleAddress], }); const whale = await ethers.getSigner(whaleAddress); // Use whale's tokens await dai .connect(whale) .transfer(addr1.address, ethers.utils.parseEther("1000")); }); ``` ## Gas Optimization Testing ```javascript const { expect } = require("chai"); describe("Gas Optimization", function () { it("Compare gas usage between implementations", async function () { const Implementation1 = await ethers.getContractFactory("OptimizedContract"); const Implementation2 = await ethers.getContractFactory( "UnoptimizedContract", ); const contract1 = await Implementation1.deploy(); const contract2 = await Implementation2.deploy(); const tx1 = await contract1.doSomething(); const receipt1 = await tx1.wait(); const tx2 = await contract2.doSomething(); const receipt2 = await tx2.wait(); console.log("Optimized gas:", receipt1.gasUsed.toString()); console.log("Unoptimized gas:", receipt2.gasUsed.toString()); expect(receipt1.gasUsed).to.be.lessThan(receipt2.gasUsed); }); }); ``` ## Coverage Reporting ```bash # Generate coverage report npx hardhat coverage # Output shows: # File | % Stmts | % Branch | % Funcs | % Lines | # -------------------|---------|----------|---------|---------| # contracts/Token.sol | 100 | 90 | 100 | 95 | ``` ## Contract Verification ```javascript // Verify on Etherscan await hre.run("verify:verify", { address: contractAddress, constructorArguments: [arg1, arg2], }); ``` ```bash # Or via CLI npx hardhat verify --network mainnet CONTRACT_ADDRESS "Constructor arg1" "arg2" ``` ## CI/CD Integration ```yaml # .github/workflows/test.yml name: Tests on: [push, pull_request] jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - uses: actions/setup-node@v2 with: node-version: "16" - run: npm install - run: npx hardhat compile - run: npx hardhat test - run: npx hardhat coverage - name: Upload coverage to Codecov uses: codecov/codecov-action@v2 ``` ## Resources - **references/hardhat-setup.md**: Hardhat configuration guide - **references/foundry-setup.md**: Foundry testing framework - **references/test-patterns.md**: Testing best practices - **references/mainnet-forking.md**: Fork testing strategies - **references/contract-verification.md**: Etherscan verification - **assets/hardhat-config.js**: Complete Hardhat configuration - **assets/test-suite.js**: Comprehensive test examples - **assets/foundry.toml**: Foundry configuration - **scripts/test-contract.sh**: Automated testing script ## Best Practices 1. **Test Coverage**: Aim for >90% coverage 2. **Edge Cases**: Test boundary conditions 3. **Gas Limits**: Verify functions don't hit block gas limit 4. **Reentrancy**: Test for reentrancy vulnerabilities 5. **Access Control**: Test unauthorized access attempts 6. **Events**: Verify event emissions 7. **Fixtures**: Use fixtures to avoid code duplication 8. **Mainnet Fork**: Test with real contracts 9. **Fuzzing**: Use property-based testing 10. **CI/CD**: Automate testing on every commit
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

kpi-dashboard-design

Design effective KPI dashboards with metrics selection,

coding
⭐1
# KPI Dashboard Design Comprehensive patterns for designing effective Key Performance Indicator (KPI) dashboards that drive business decisions. ## When to Use This Skill - Designing executive dashboards - Selecting meaningful KPIs - Building real-time monitoring displays - Creating department-specific metrics views - Improving existing dashboard layouts - Establishing metric governance ## Core Concepts ### 1. KPI Framework | Level | Focus | Update Frequency | Audience | | --------------- | ---------------- | ----------------- | ---------- | | **Strategic** | Long-term goals | Monthly/Quarterly | Executives | | **Tactical** | Department goals | Weekly/Monthly | Managers | | **Operational** | Day-to-day | Real-time/Daily | Teams | ### 2. SMART KPIs ``` Specific: Clear definition Measurable: Quantifiable Achievable: Realistic targets Relevant: Aligned to goals Time-bound: Defined period ``` ### 3. Dashboard Hierarchy ``` β”œβ”€β”€ Executive Summary (1 page) β”‚ β”œβ”€β”€ 4-6 headline KPIs β”‚ β”œβ”€β”€ Trend indicators β”‚ └── Key alerts β”œβ”€β”€ Department Views β”‚ β”œβ”€β”€ Sales Dashboard β”‚ β”œβ”€β”€ Marketing Dashboard β”‚ β”œβ”€β”€ Operations Dashboard β”‚ └── Finance Dashboard └── Detailed Drilldowns β”œβ”€β”€ Individual metrics └── Root cause analysis ``` ## Common KPIs by Department ### Sales KPIs ```yaml Revenue Metrics: - Monthly Recurring Revenue (MRR) - Annual Recurring Revenue (ARR) - Average Revenue Per User (ARPU) - Revenue Growth Rate Pipeline Metrics: - Sales Pipeline Value - Win Rate - Average Deal Size - Sales Cycle Length Activity Metrics: - Calls/Emails per Rep - Demos Scheduled - Proposals Sent - Close Rate ``` ### Marketing KPIs ```yaml Acquisition: - Cost Per Acquisition (CPA) - Customer Acquisition Cost (CAC) - Lead Volume - Marketing Qualified Leads (MQL) Engagement: - Website Traffic - Conversion Rate - Email Open/Click Rate - Social Engagement ROI: - Marketing ROI - Campaign Performance - Channel Attribution - CAC Payback Period ``` ### Product KPIs ```yaml Usage: - Daily/Monthly Active Users (DAU/MAU) - Session Duration - Feature Adoption Rate - Stickiness (DAU/MAU) Quality: - Net Promoter Score (NPS) - Customer Satisfaction (CSAT) - Bug/Issue Count - Time to Resolution Growth: - User Growth Rate - Activation Rate - Retention Rate - Churn Rate ``` ### Finance KPIs ```yaml Profitability: - Gross Margin - Net Profit Margin - EBITDA - Operating Margin Liquidity: - Current Ratio - Quick Ratio - Cash Flow - Working Capital Efficiency: - Revenue per Employee - Operating Expense Ratio - Days Sales Outstanding - Inventory Turnover ``` ## Dashboard Layout Patterns ### Pattern 1: Executive Summary ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ EXECUTIVE DASHBOARD [Date Range β–Ό] β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ REVENUE β”‚ PROFIT β”‚ CUSTOMERS β”‚ NPS SCORE β”‚ β”‚ $2.4M β”‚ $450K β”‚ 12,450 β”‚ 72 β”‚ β”‚ β–² 12% β”‚ β–² 8% β”‚ β–² 15% β”‚ β–² 5pts β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ β”‚ β”‚ Revenue Trend β”‚ Revenue by Product β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ /\ /\ β”‚ β”‚ β”‚ β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ 45% β”‚ β”‚ β”‚ β”‚ / \ / \ /\ β”‚ β”‚ β”‚ β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ 32% β”‚ β”‚ β”‚ β”‚ / \/ \ / \ β”‚ β”‚ β”‚ β–ˆβ–ˆβ–ˆβ–ˆ 18% β”‚ β”‚ β”‚ β”‚ / \/ \ β”‚ β”‚ β”‚ β–ˆβ–ˆ 5% β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ πŸ”΄ Alert: Churn rate exceeded threshold (>5%) β”‚ β”‚ 🟑 Warning: Support ticket volume 20% above average β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` ### Pattern 2: SaaS Metrics Dashboard ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ SAAS METRICS Jan 2024 [Monthly β–Ό] β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ MRR GROWTH β”‚ β”‚ β”‚ MRR β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ $125,000 β”‚ β”‚ β”‚ /── β”‚ β”‚ β”‚ β”‚ β–² 8% β”‚ β”‚ β”‚ /────/ β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ /────/ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ /────/ β”‚ β”‚ β”‚ β”‚ ARR β”‚ β”‚ β”‚ /────/ β”‚ β”‚ β”‚ β”‚ $1,500,000 β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β–² 15% β”‚ β”‚ J F M A M J J A S O N D β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ UNIT ECONOMICS β”‚ COHORT RETENTION β”‚ β”‚ β”‚ β”‚ β”‚ CAC: $450 β”‚ Month 1: β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ 100% β”‚ β”‚ LTV: $2,700 β”‚ Month 3: β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ 85% β”‚ β”‚ LTV/CAC: 6.0x β”‚ Month 6: β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ 80% β”‚ β”‚ β”‚ Month 12: β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ 72% β”‚ β”‚ Payback: 4 months β”‚ β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ CHURN ANALYSIS β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ Gross β”‚ Net β”‚ Logo β”‚ Expansion β”‚ β”‚ β”‚ β”‚ 4.2% β”‚ 1.8% β”‚ 3.1% β”‚ 2.4% β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` ### Pattern 3: Real-time Operations ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ OPERATIONS CENTER Live ● Last: 10:42:15 β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ SYSTEM HEALTH β”‚ SERVICE STATUS β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ β”‚ CPU MEM DISK β”‚ β”‚ ● API Gateway Healthy β”‚ β”‚ β”‚ 45% 72% 58% β”‚ β”‚ ● User Service Healthy β”‚ β”‚ β”‚ β–ˆβ–ˆβ–ˆ β–ˆβ–ˆβ–ˆβ–ˆ β–ˆβ–ˆβ–ˆ β”‚ β”‚ ● Payment Service Degraded β”‚ β”‚ β”‚ β–ˆβ–ˆβ–ˆ β–ˆβ–ˆβ–ˆβ–ˆ β–ˆβ–ˆβ–ˆ β”‚ β”‚ ● Database Healthy β”‚ β”‚ β”‚ β–ˆβ–ˆβ–ˆ β–ˆβ–ˆβ–ˆβ–ˆ β–ˆβ–ˆβ–ˆ β”‚ β”‚ ● Cache Healthy β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ REQUEST THROUGHPUT β”‚ ERROR RATE β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ β–β–‚β–ƒβ–„β–…β–†β–‡β–ˆβ–‡β–†β–…β–„β–ƒβ–‚β–β–‚β–ƒβ–„β–… β”‚ β”‚ β”‚ ▁▁▁▁▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁ β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ Current: 12,450 req/s β”‚ Current: 0.02% β”‚ β”‚ Peak: 18,200 req/s β”‚ Threshold: 1.0% β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ RECENT ALERTS β”‚ β”‚ 10:40 🟑 High latency on payment-service (p99 > 500ms) β”‚ β”‚ 10:35 🟒 Resolved: Database connection pool recovered β”‚ β”‚ 10:22 πŸ”΄ Payment service circuit breaker tripped β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` ## Implementation Patterns ### SQL for KPI Calculations ```sql -- Monthly Recurring Revenue (MRR) WITH mrr_calculation AS ( SELECT DATE_TRUNC('month', billing_date) AS month, SUM( CASE subscription_interval WHEN 'monthly' THEN amount WHEN 'yearly' THEN amount / 12 WHEN 'quarterly' THEN amount / 3 END ) AS mrr FROM subscriptions WHERE status = 'active' GROUP BY DATE_TRUNC('month', billing_date) ) SELECT month, mrr, LAG(mrr) OVER (ORDER BY month) AS prev_mrr, (mrr - LAG(mrr) OVER (ORDER BY month)) / LAG(mrr) OVER (ORDER BY month) * 100 AS growth_pct FROM mrr_calculation; -- Cohort Retention WITH cohorts AS ( SELECT user_id, DATE_TRUNC('month', created_at) AS cohort_month FROM users ), activity AS ( SELECT user_id, DATE_TRUNC('month', event_date) AS activity_month FROM user_events WHERE event_type = 'active_session' ) SELECT c.cohort_month, EXTRACT(MONTH FROM age(a.activity_month, c.cohort_month)) AS months_since_signup, COUNT(DISTINCT a.user_id) AS active_users, COUNT(DISTINCT a.user_id)::FLOAT / COUNT(DISTINCT c.user_id) * 100 AS retention_rate FROM cohorts c LEFT JOIN activity a ON c.user_id = a.user_id AND a.activity_month >= c.cohort_month GROUP BY c.cohort_month, EXTRACT(MONTH FROM age(a.activity_month, c.cohort_month)) ORDER BY c.cohort_month, months_since_signup; -- Customer Acquisition Cost (CAC) SELECT DATE_TRUNC('month', acquired_date) AS month, SUM(marketing_spend) / NULLIF(COUNT(new_customers), 0) AS cac, SUM(marketing_spend) AS total_spend, COUNT(new_customers) AS customers_acquired FROM ( SELECT DATE_TRUNC('month', u.created_at) AS acquired_date, u.id AS new_customers, m.spend AS marketing_spend FROM users u JOIN marketing_spend m ON DATE_TRUNC('month', u.created_at) = m.month WHERE u.source = 'marketing' ) acquisition GROUP BY DATE_TRUNC('month', acquired_date); ``` ### Python Dashboard Code (Streamlit) ```python import streamlit as st import pandas as pd import plotly.express as px import plotly.graph_objects as go st.set_page_config(page_title="KPI Dashboard", layout="wide") # Header with date filter col1, col2 = st.columns([3, 1]) with col1: st.title("Executive Dashboard") with col2: date_range = st.selectbox( "Period", ["Last 7 Days", "Last 30 Days", "Last Quarter", "YTD"] ) # KPI Cards def metric_card(label, value, delta, prefix="", suffix=""): delta_color = "green" if delta >= 0 else "red" delta_arrow = "β–²" if delta >= 0 else "β–Ό" st.metric( label=label, value=f"{prefix}{value:,.0f}{suffix}", delta=f"{delta_arrow} {abs(delta):.1f}%" ) col1, col2, col3, col4 = st.columns(4) with col1: metric_card("Revenue", 2400000, 12.5, prefix="$") with col2: metric_card("Customers", 12450, 15.2) with col3: metric_card("NPS Score", 72, 5.0) with col4: metric_card("Churn Rate", 4.2, -0.8, suffix="%") # Charts col1, col2 = st.columns(2) with col1: st.subheader("Revenue Trend") revenue_data = pd.DataFrame({ 'Month': pd.date_range('2024-01-01', periods=12, freq='M'), 'Revenue': [180000, 195000, 210000, 225000, 240000, 255000, 270000, 285000, 300000, 315000, 330000, 345000] }) fig = px.line(revenue_data, x='Month', y='Revenue', line_shape='spline', markers=True) fig.update_layout(height=300) st.plotly_chart(fig, use_container_width=True) with col2: st.subheader("Revenue by Product") product_data = pd.DataFrame({ 'Product': ['Enterprise', 'Professional', 'Starter', 'Other'], 'Revenue': [45, 32, 18, 5] }) fig = px.pie(product_data, values='Revenue', names='Product', hole=0.4) fig.update_layout(height=300) st.plotly_chart(fig, use_container_width=True) # Cohort Heatmap st.subheader("Cohort Retention") cohort_data = pd.DataFrame({ 'Cohort': ['Jan', 'Feb', 'Mar', 'Apr', 'May'], 'M0': [100, 100, 100, 100, 100], 'M1': [85, 87, 84, 86, 88], 'M2': [78, 80, 76, 79, None], 'M3': [72, 74, 70, None, None], 'M4': [68, 70, None, None, None], }) fig = go.Figure(data=go.Heatmap( z=cohort_data.iloc[:, 1:].values, x=['M0', 'M1', 'M2', 'M3', 'M4'], y=cohort_data['Cohort'], colorscale='Blues', text=cohort_data.iloc[:, 1:].values, texttemplate='%{text}%', textfont={"size": 12}, )) fig.update_layout(height=250) st.plotly_chart(fig, use_container_width=True) # Alerts Section st.subheader("Alerts") alerts = [ {"level": "error", "message": "Churn rate exceeded threshold (>5%)"}, {"level": "warning", "message": "Support ticket volume 20% above average"}, ] for alert in alerts: if alert["level"] == "error": st.error(f"πŸ”΄ {alert['message']}") elif alert["level"] == "warning": st.warning(f"🟑 {alert['message']}") ``` ## Best Practices ### Do's - **Limit to 5-7 KPIs** - Focus on what matters - **Show context** - Comparisons, trends, targets - **Use consistent colors** - Red=bad, green=good - **Enable drilldown** - From summary to detail - **Update appropriately** - Match metric frequency ### Don'ts - **Don't show vanity metrics** - Focus on actionable data - **Don't overcrowd** - White space aids comprehension - **Don't use 3D charts** - They distort perception - **Don't hide methodology** - Document calculations - **Don't ignore mobile** - Ensure responsive design ## Resources - [Stephen Few's Dashboard Design](https://www.perceptualedge.com/articles/visual_business_intelligence/rules_for_using_color.pdf) - [Edward Tufte's Principles](https://www.edwardtufte.com/tufte/) - [Google Data Studio Gallery](https://datastudio.google.com/gallery)
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

deployment-pipeline-design

Design multi-stage CI/CD pipelines with approval gates, security

coding
⭐1
# Deployment Pipeline Design Architecture patterns for multi-stage CI/CD pipelines with approval gates and deployment strategies. ## Purpose Design robust, secure deployment pipelines that balance speed with safety through proper stage organization and approval workflows. ## When to Use - Design CI/CD architecture - Implement deployment gates - Configure multi-environment pipelines - Establish deployment best practices - Implement progressive delivery ## Pipeline Stages ### Standard Pipeline Flow ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Build β”‚ β†’ β”‚ Test β”‚ β†’ β”‚ Staging β”‚ β†’ β”‚ Approveβ”‚ β†’ β”‚Productionβ”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` ### Detailed Stage Breakdown 1. **Source** - Code checkout 2. **Build** - Compile, package, containerize 3. **Test** - Unit, integration, security scans 4. **Staging Deploy** - Deploy to staging environment 5. **Integration Tests** - E2E, smoke tests 6. **Approval Gate** - Manual approval required 7. **Production Deploy** - Canary, blue-green, rolling 8. **Verification** - Health checks, monitoring 9. **Rollback** - Automated rollback on failure ## Approval Gate Patterns ### Pattern 1: Manual Approval ```yaml # GitHub Actions production-deploy: needs: staging-deploy environment: name: production url: https://app.example.com runs-on: ubuntu-latest steps: - name: Deploy to production run: | # Deployment commands ``` ### Pattern 2: Time-Based Approval ```yaml # GitLab CI deploy:production: stage: deploy script: - deploy.sh production environment: name: production when: delayed start_in: 30 minutes only: - main ``` ### Pattern 3: Multi-Approver ```yaml # Azure Pipelines stages: - stage: Production dependsOn: Staging jobs: - deployment: Deploy environment: name: production resourceType: Kubernetes strategy: runOnce: preDeploy: steps: - task: ManualValidation@0 inputs: notifyUsers: "team-leads@example.com" instructions: "Review staging metrics before approving" ``` **Reference:** See `assets/approval-gate-template.yml` ## Deployment Strategies ### 1. Rolling Deployment ```yaml apiVersion: apps/v1 kind: Deployment metadata: name: my-app spec: replicas: 10 strategy: type: RollingUpdate rollingUpdate: maxSurge: 2 maxUnavailable: 1 ``` **Characteristics:** - Gradual rollout - Zero downtime - Easy rollback - Best for most applications ### 2. Blue-Green Deployment ```yaml # Blue (current) kubectl apply -f blue-deployment.yaml kubectl label service my-app version=blue # Green (new) kubectl apply -f green-deployment.yaml # Test green environment kubectl label service my-app version=green # Rollback if needed kubectl label service my-app version=blue ``` **Characteristics:** - Instant switchover - Easy rollback - Doubles infrastructure cost temporarily - Good for high-risk deployments ### 3. Canary Deployment ```yaml apiVersion: argoproj.io/v1alpha1 kind: Rollout metadata: name: my-app spec: replicas: 10 strategy: canary: steps: - setWeight: 10 - pause: { duration: 5m } - setWeight: 25 - pause: { duration: 5m } - setWeight: 50 - pause: { duration: 5m } - setWeight: 100 ``` **Characteristics:** - Gradual traffic shift - Risk mitigation - Real user testing - Requires service mesh or similar ### 4. Feature Flags ```python from flagsmith import Flagsmith flagsmith = Flagsmith(environment_key="API_KEY") if flagsmith.has_feature("new_checkout_flow"): # New code path process_checkout_v2() else: # Existing code path process_checkout_v1() ``` **Characteristics:** - Deploy without releasing - A/B testing - Instant rollback - Granular control ## Pipeline Orchestration ### Multi-Stage Pipeline Example ```yaml name: Production Pipeline on: push: branches: [main] jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Build application run: make build - name: Build Docker image run: docker build -t myapp:${{ github.sha }} . - name: Push to registry run: docker push myapp:${{ github.sha }} test: needs: build runs-on: ubuntu-latest steps: - name: Unit tests run: make test - name: Security scan run: trivy image myapp:${{ github.sha }} deploy-staging: needs: test runs-on: ubuntu-latest environment: name: staging steps: - name: Deploy to staging run: kubectl apply -f k8s/staging/ integration-test: needs: deploy-staging runs-on: ubuntu-latest steps: - name: Run E2E tests run: npm run test:e2e deploy-production: needs: integration-test runs-on: ubuntu-latest environment: name: production steps: - name: Canary deployment run: | kubectl apply -f k8s/production/ kubectl argo rollouts promote my-app verify: needs: deploy-production runs-on: ubuntu-latest steps: - name: Health check run: curl -f https://app.example.com/health - name: Notify team run: | curl -X POST ${{ secrets.SLACK_WEBHOOK }} \ -d '{"text":"Production deployment successful!"}' ``` ## Pipeline Best Practices 1. **Fail fast** - Run quick tests first 2. **Parallel execution** - Run independent jobs concurrently 3. **Caching** - Cache dependencies between runs 4. **Artifact management** - Store build artifacts 5. **Environment parity** - Keep environments consistent 6. **Secrets management** - Use secret stores (Vault, etc.) 7. **Deployment windows** - Schedule deployments appropriately 8. **Monitoring integration** - Track deployment metrics 9. **Rollback automation** - Auto-rollback on failures 10. **Documentation** - Document pipeline stages ## Rollback Strategies ### Automated Rollback ```yaml deploy-and-verify: steps: - name: Deploy new version run: kubectl apply -f k8s/ - name: Wait for rollout run: kubectl rollout status deployment/my-app - name: Health check id: health run: | for i in {1..10}; do if curl -sf https://app.example.com/health; then exit 0 fi sleep 10 done exit 1 - name: Rollback on failure if: failure() run: kubectl rollout undo deployment/my-app ``` ### Manual Rollback ```bash # List revision history kubectl rollout history deployment/my-app # Rollback to previous version kubectl rollout undo deployment/my-app # Rollback to specific revision kubectl rollout undo deployment/my-app --to-revision=3 ``` ## Monitoring and Metrics ### Key Pipeline Metrics - **Deployment Frequency** - How often deployments occur - **Lead Time** - Time from commit to production - **Change Failure Rate** - Percentage of failed deployments - **Mean Time to Recovery (MTTR)** - Time to recover from failure - **Pipeline Success Rate** - Percentage of successful runs - **Average Pipeline Duration** - Time to complete pipeline ### Integration with Monitoring ```yaml - name: Post-deployment verification run: | # Wait for metrics stabilization sleep 60 # Check error rate ERROR_RATE=$(curl -s "$PROMETHEUS_URL/api/v1/query?query=rate(http_errors_total[5m])" | jq '.data.result[0].value[1]') if (( $(echo "$ERROR_RATE > 0.01" | bc -l) )); then echo "Error rate too high: $ERROR_RATE" exit 1 fi ``` ## Reference Files - `references/pipeline-orchestration.md` - Complex pipeline patterns - `assets/approval-gate-template.yml` - Approval workflow templates ## Related Skills - `github-actions-templates` - For GitHub Actions implementation - `gitlab-ci-patterns` - For GitLab CI implementation - `secrets-management` - For secrets handling
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

github-actions-templates

Create production-ready GitHub Actions workflows for automated

coding
⭐1
# GitHub Actions Templates Production-ready GitHub Actions workflow patterns for testing, building, and deploying applications. ## Purpose Create efficient, secure GitHub Actions workflows for continuous integration and deployment across various tech stacks. ## When to Use - Automate testing and deployment - Build Docker images and push to registries - Deploy to Kubernetes clusters - Run security scans - Implement matrix builds for multiple environments ## Common Workflow Patterns ### Pattern 1: Test Workflow ```yaml name: Test on: push: branches: [main, develop] pull_request: branches: [main] jobs: test: runs-on: ubuntu-latest strategy: matrix: node-version: [18.x, 20.x] steps: - uses: actions/checkout@v4 - name: Use Node.js ${{ matrix.node-version }} uses: actions/setup-node@v4 with: node-version: ${{ matrix.node-version }} cache: "npm" - name: Install dependencies run: npm ci - name: Run linter run: npm run lint - name: Run tests run: npm test - name: Upload coverage uses: codecov/codecov-action@v3 with: files: ./coverage/lcov.info ``` **Reference:** See `assets/test-workflow.yml` ### Pattern 2: Build and Push Docker Image ```yaml name: Build and Push on: push: branches: [main] tags: ["v*"] env: REGISTRY: ghcr.io IMAGE_NAME: ${{ github.repository }} jobs: build: runs-on: ubuntu-latest permissions: contents: read packages: write steps: - uses: actions/checkout@v4 - name: Log in to Container Registry uses: docker/login-action@v3 with: registry: ${{ env.REGISTRY }} username: ${{ github.actor }} password: ${{ secrets.GITHUB_TOKEN }} - name: Extract metadata id: meta uses: docker/metadata-action@v5 with: images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} tags: | type=ref,event=branch type=ref,event=pr type=semver,pattern={{version}} type=semver,pattern={{major}}.{{minor}} - name: Build and push uses: docker/build-push-action@v5 with: context: . push: true tags: ${{ steps.meta.outputs.tags }} labels: ${{ steps.meta.outputs.labels }} cache-from: type=gha cache-to: type=gha,mode=max ``` **Reference:** See `assets/deploy-workflow.yml` ### Pattern 3: Deploy to Kubernetes ```yaml name: Deploy to Kubernetes on: push: branches: [main] jobs: deploy: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Configure AWS credentials uses: aws-actions/configure-aws-credentials@v4 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: us-west-2 - name: Update kubeconfig run: | aws eks update-kubeconfig --name production-cluster --region us-west-2 - name: Deploy to Kubernetes run: | kubectl apply -f k8s/ kubectl rollout status deployment/my-app -n production kubectl get services -n production - name: Verify deployment run: | kubectl get pods -n production kubectl describe deployment my-app -n production ``` ### Pattern 4: Matrix Build ```yaml name: Matrix Build on: [push, pull_request] jobs: build: runs-on: ${{ matrix.os }} strategy: matrix: os: [ubuntu-latest, macos-latest, windows-latest] python-version: ["3.9", "3.10", "3.11", "3.12"] steps: - uses: actions/checkout@v4 - name: Set up Python uses: actions/setup-python@v5 with: python-version: ${{ matrix.python-version }} - name: Install dependencies run: | python -m pip install --upgrade pip pip install -r requirements.txt - name: Run tests run: pytest ``` **Reference:** See `assets/matrix-build.yml` ## Workflow Best Practices 1. **Use specific action versions** (@v4, not @latest) 2. **Cache dependencies** to speed up builds 3. **Use secrets** for sensitive data 4. **Implement status checks** on PRs 5. **Use matrix builds** for multi-version testing 6. **Set appropriate permissions** 7. **Use reusable workflows** for common patterns 8. **Implement approval gates** for production 9. **Add notification steps** for failures 10. **Use self-hosted runners** for sensitive workloads ## Reusable Workflows ```yaml # .github/workflows/reusable-test.yml name: Reusable Test Workflow on: workflow_call: inputs: node-version: required: true type: string secrets: NPM_TOKEN: required: true jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: ${{ inputs.node-version }} - run: npm ci - run: npm test ``` **Use reusable workflow:** ```yaml jobs: call-test: uses: ./.github/workflows/reusable-test.yml with: node-version: "20.x" secrets: NPM_TOKEN: ${{ secrets.NPM_TOKEN }} ``` ## Security Scanning ```yaml name: Security Scan on: push: branches: [main] pull_request: branches: [main] jobs: security: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Run Trivy vulnerability scanner uses: aquasecurity/trivy-action@master with: scan-type: "fs" scan-ref: "." format: "sarif" output: "trivy-results.sarif" - name: Upload Trivy results to GitHub Security uses: github/codeql-action/upload-sarif@v2 with: sarif_file: "trivy-results.sarif" - name: Run Snyk Security Scan uses: snyk/actions/node@master env: SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }} ``` ## Deployment with Approvals ```yaml name: Deploy to Production on: push: tags: ["v*"] jobs: deploy: runs-on: ubuntu-latest environment: name: production url: https://app.example.com steps: - uses: actions/checkout@v4 - name: Deploy application run: | echo "Deploying to production..." # Deployment commands here - name: Notify Slack if: success() uses: slackapi/slack-github-action@v1 with: webhook-url: ${{ secrets.SLACK_WEBHOOK }} payload: | { "text": "Deployment to production completed successfully!" } ``` ## Reference Files - `assets/test-workflow.yml` - Testing workflow template - `assets/deploy-workflow.yml` - Deployment workflow template - `assets/matrix-build.yml` - Matrix build template - `references/common-workflows.md` - Common workflow patterns ## Related Skills - `gitlab-ci-patterns` - For GitLab CI workflows - `deployment-pipeline-design` - For pipeline architecture - `secrets-management` - For secrets handling
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

cost-optimization

Optimize cloud costs through resource rightsizing, tagging

architecture
⭐1
# Cloud Cost Optimization Strategies and patterns for optimizing cloud costs across AWS, Azure, and GCP. ## Purpose Implement systematic cost optimization strategies to reduce cloud spending while maintaining performance and reliability. ## When to Use - Reduce cloud spending - Right-size resources - Implement cost governance - Optimize multi-cloud costs - Meet budget constraints ## Cost Optimization Framework ### 1. Visibility - Implement cost allocation tags - Use cloud cost management tools - Set up budget alerts - Create cost dashboards ### 2. Right-Sizing - Analyze resource utilization - Downsize over-provisioned resources - Use auto-scaling - Remove idle resources ### 3. Pricing Models - Use reserved capacity - Leverage spot/preemptible instances - Implement savings plans - Use committed use discounts ### 4. Architecture Optimization - Use managed services - Implement caching - Optimize data transfer - Use lifecycle policies ## AWS Cost Optimization ### Reserved Instances ``` Savings: 30-72% vs On-Demand Term: 1 or 3 years Payment: All/Partial/No upfront Flexibility: Standard or Convertible ``` ### Savings Plans ``` Compute Savings Plans: 66% savings EC2 Instance Savings Plans: 72% savings Applies to: EC2, Fargate, Lambda Flexible across: Instance families, regions, OS ``` ### Spot Instances ``` Savings: Up to 90% vs On-Demand Best for: Batch jobs, CI/CD, stateless workloads Risk: 2-minute interruption notice Strategy: Mix with On-Demand for resilience ``` ### S3 Cost Optimization ```hcl resource "aws_s3_bucket_lifecycle_configuration" "example" { bucket = aws_s3_bucket.example.id rule { id = "transition-to-ia" status = "Enabled" transition { days = 30 storage_class = "STANDARD_IA" } transition { days = 90 storage_class = "GLACIER" } expiration { days = 365 } } } ``` ## Azure Cost Optimization ### Reserved VM Instances - 1 or 3 year terms - Up to 72% savings - Flexible sizing - Exchangeable ### Azure Hybrid Benefit - Use existing Windows Server licenses - Up to 80% savings with RI - Available for Windows and SQL Server ### Azure Advisor Recommendations - Right-size VMs - Delete unused resources - Use reserved capacity - Optimize storage ## GCP Cost Optimization ### Committed Use Discounts - 1 or 3 year commitment - Up to 57% savings - Applies to vCPUs and memory - Resource-based or spend-based ### Sustained Use Discounts - Automatic discounts - Up to 30% for running instances - No commitment required - Applies to Compute Engine, GKE ### Preemptible VMs - Up to 80% savings - 24-hour maximum runtime - Best for batch workloads ## Tagging Strategy ### AWS Tagging ```hcl locals { common_tags = { Environment = "production" Project = "my-project" CostCenter = "engineering" Owner = "team@example.com" ManagedBy = "terraform" } } resource "aws_instance" "example" { ami = "ami-12345678" instance_type = "t3.medium" tags = merge( local.common_tags, { Name = "web-server" } ) } ``` **Reference:** See `references/tagging-standards.md` ## Cost Monitoring ### Budget Alerts ```hcl # AWS Budget resource "aws_budgets_budget" "monthly" { name = "monthly-budget" budget_type = "COST" limit_amount = "1000" limit_unit = "USD" time_period_start = "2024-01-01_00:00" time_unit = "MONTHLY" notification { comparison_operator = "GREATER_THAN" threshold = 80 threshold_type = "PERCENTAGE" notification_type = "ACTUAL" subscriber_email_addresses = ["team@example.com"] } } ``` ### Cost Anomaly Detection - AWS Cost Anomaly Detection - Azure Cost Management alerts - GCP Budget alerts ## Architecture Patterns ### Pattern 1: Serverless First - Use Lambda/Functions for event-driven - Pay only for execution time - Auto-scaling included - No idle costs ### Pattern 2: Right-Sized Databases ``` Development: t3.small RDS Staging: t3.large RDS Production: r6g.2xlarge RDS with read replicas ``` ### Pattern 3: Multi-Tier Storage ``` Hot data: S3 Standard Warm data: S3 Standard-IA (30 days) Cold data: S3 Glacier (90 days) Archive: S3 Deep Archive (365 days) ``` ### Pattern 4: Auto-Scaling ```hcl resource "aws_autoscaling_policy" "scale_up" { name = "scale-up" scaling_adjustment = 2 adjustment_type = "ChangeInCapacity" cooldown = 300 autoscaling_group_name = aws_autoscaling_group.main.name } resource "aws_cloudwatch_metric_alarm" "cpu_high" { alarm_name = "cpu-high" comparison_operator = "GreaterThanThreshold" evaluation_periods = "2" metric_name = "CPUUtilization" namespace = "AWS/EC2" period = "60" statistic = "Average" threshold = "80" alarm_actions = [aws_autoscaling_policy.scale_up.arn] } ``` ## Cost Optimization Checklist - [ ] Implement cost allocation tags - [ ] Delete unused resources (EBS, EIPs, snapshots) - [ ] Right-size instances based on utilization - [ ] Use reserved capacity for steady workloads - [ ] Implement auto-scaling - [ ] Optimize storage classes - [ ] Use lifecycle policies - [ ] Enable cost anomaly detection - [ ] Set budget alerts - [ ] Review costs weekly - [ ] Use spot/preemptible instances - [ ] Optimize data transfer costs - [ ] Implement caching layers - [ ] Use managed services - [ ] Monitor and optimize continuously ## Tools - **AWS:** Cost Explorer, Cost Anomaly Detection, Compute Optimizer - **Azure:** Cost Management, Advisor - **GCP:** Cost Management, Recommender - **Multi-cloud:** CloudHealth, Cloudability, Kubecost ## Reference Files - `references/tagging-standards.md` - Tagging conventions - `assets/cost-analysis-template.xlsx` - Cost analysis spreadsheet ## Related Skills - `terraform-module-library` - For resource provisioning - `multi-cloud-architecture` - For cloud selection
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

screen-reader-testing

Test web applications with screen readers including VoiceOver,

coding
⭐1
# Screen Reader Testing Practical guide to testing web applications with screen readers for comprehensive accessibility validation. ## When to Use This Skill - Validating screen reader compatibility - Testing ARIA implementations - Debugging assistive technology issues - Verifying form accessibility - Testing dynamic content announcements - Ensuring navigation accessibility ## Core Concepts ### 1. Major Screen Readers | Screen Reader | Platform | Browser | Usage | | ------------- | --------- | -------------- | ----- | | **VoiceOver** | macOS/iOS | Safari | ~15% | | **NVDA** | Windows | Firefox/Chrome | ~31% | | **JAWS** | Windows | Chrome/IE | ~40% | | **TalkBack** | Android | Chrome | ~10% | | **Narrator** | Windows | Edge | ~4% | ### 2. Testing Priority ``` Minimum Coverage: 1. NVDA + Firefox (Windows) 2. VoiceOver + Safari (macOS) 3. VoiceOver + Safari (iOS) Comprehensive Coverage: + JAWS + Chrome (Windows) + TalkBack + Chrome (Android) + Narrator + Edge (Windows) ``` ### 3. Screen Reader Modes | Mode | Purpose | When Used | | ------------------ | ---------------------- | ----------------- | | **Browse/Virtual** | Read content | Default reading | | **Focus/Forms** | Interact with controls | Filling forms | | **Application** | Custom widgets | ARIA applications | ## VoiceOver (macOS) ### Setup ``` Enable: System Preferences β†’ Accessibility β†’ VoiceOver Toggle: Cmd + F5 Quick Toggle: Triple-press Touch ID ``` ### Essential Commands ``` Navigation: VO = Ctrl + Option (VoiceOver modifier) VO + Right Arrow Next element VO + Left Arrow Previous element VO + Shift + Down Enter group VO + Shift + Up Exit group Reading: VO + A Read all from cursor Ctrl Stop speaking VO + B Read current paragraph Interaction: VO + Space Activate element VO + Shift + M Open menu Tab Next focusable element Shift + Tab Previous focusable element Rotor (VO + U): Navigate by: Headings, Links, Forms, Landmarks Left/Right Arrow Change rotor category Up/Down Arrow Navigate within category Enter Go to item Web Specific: VO + Cmd + H Next heading VO + Cmd + J Next form control VO + Cmd + L Next link VO + Cmd + T Next table ``` ### Testing Checklist ```markdown ## VoiceOver Testing Checklist ### Page Load - [ ] Page title announced - [ ] Main landmark found - [ ] Skip link works ### Navigation - [ ] All headings discoverable via rotor - [ ] Heading levels logical (H1 β†’ H2 β†’ H3) - [ ] Landmarks properly labeled - [ ] Skip links functional ### Links & Buttons - [ ] Link purpose clear - [ ] Button actions described - [ ] New window/tab announced ### Forms - [ ] All labels read with inputs - [ ] Required fields announced - [ ] Error messages read - [ ] Instructions available - [ ] Focus moves to errors ### Dynamic Content - [ ] Alerts announced immediately - [ ] Loading states communicated - [ ] Content updates announced - [ ] Modals trap focus correctly ### Tables - [ ] Headers associated with cells - [ ] Table navigation works - [ ] Complex tables have captions ``` ### Common Issues & Fixes ```html <!-- Issue: Button not announcing purpose --> <button><svg>...</svg></button> <!-- Fix --> <button aria-label="Close dialog"><svg aria-hidden="true">...</svg></button> <!-- Issue: Dynamic content not announced --> <div id="results">New results loaded</div> <!-- Fix --> <div id="results" role="status" aria-live="polite">New results loaded</div> <!-- Issue: Form error not read --> <input type="email" /> <span class="error">Invalid email</span> <!-- Fix --> <input type="email" aria-invalid="true" aria-describedby="email-error" /> <span id="email-error" role="alert">Invalid email</span> ``` ## NVDA (Windows) ### Setup ``` Download: nvaccess.org Start: Ctrl + Alt + N Stop: Insert + Q ``` ### Essential Commands ``` Navigation: Insert = NVDA modifier Down Arrow Next line Up Arrow Previous line Tab Next focusable Shift + Tab Previous focusable Reading: NVDA + Down Arrow Say all Ctrl Stop speech NVDA + Up Arrow Current line Headings: H Next heading Shift + H Previous heading 1-6 Heading level 1-6 Forms: F Next form field B Next button E Next edit field X Next checkbox C Next combo box Links: K Next link U Next unvisited link V Next visited link Landmarks: D Next landmark Shift + D Previous landmark Tables: T Next table Ctrl + Alt + Arrows Navigate cells Elements List (NVDA + F7): Shows all links, headings, form fields, landmarks ``` ### Browse vs Focus Mode ``` NVDA automatically switches modes: - Browse Mode: Arrow keys navigate content - Focus Mode: Arrow keys control interactive elements Manual switch: NVDA + Space Watch for: - "Browse mode" announcement when navigating - "Focus mode" when entering form fields - Application role forces forms mode ``` ### Testing Script ```markdown ## NVDA Test Script ### Initial Load 1. Navigate to page 2. Let page finish loading 3. Press Insert + Down to read all 4. Note: Page title, main content identified? ### Landmark Navigation 1. Press D repeatedly 2. Check: All main areas reachable? 3. Check: Landmarks properly labeled? ### Heading Navigation 1. Press Insert + F7 β†’ Headings 2. Check: Logical heading structure? 3. Press H to navigate headings 4. Check: All sections discoverable? ### Form Testing 1. Press F to find first form field 2. Check: Label read? 3. Fill in invalid data 4. Submit form 5. Check: Errors announced? 6. Check: Focus moved to error? ### Interactive Elements 1. Tab through all interactive elements 2. Check: Each announces role and state 3. Activate buttons with Enter/Space 4. Check: Result announced? ### Dynamic Content 1. Trigger content update 2. Check: Change announced? 3. Open modal 4. Check: Focus trapped? 5. Close modal 6. Check: Focus returns? ``` ## JAWS (Windows) ### Essential Commands ``` Start: Desktop shortcut or Ctrl + Alt + J Virtual Cursor: Auto-enabled in browsers Navigation: Arrow keys Navigate content Tab Next focusable Insert + Down Read all Ctrl Stop speech Quick Keys: H Next heading T Next table F Next form field B Next button G Next graphic L Next list ; Next landmark Forms Mode: Enter Enter forms mode Numpad + Exit forms mode F5 List form fields Lists: Insert + F7 Link list Insert + F6 Heading list Insert + F5 Form field list Tables: Ctrl + Alt + Arrows Table navigation ``` ## TalkBack (Android) ### Setup ``` Enable: Settings β†’ Accessibility β†’ TalkBack Toggle: Hold both volume buttons 3 seconds ``` ### Gestures ``` Explore: Drag finger across screen Next: Swipe right Previous: Swipe left Activate: Double tap Scroll: Two finger swipe Reading Controls (swipe up then right): - Headings - Links - Controls - Characters - Words - Lines - Paragraphs ``` ## Common Test Scenarios ### 1. Modal Dialog ```html <!-- Accessible modal structure --> <div role="dialog" aria-modal="true" aria-labelledby="dialog-title" aria-describedby="dialog-desc" > <h2 id="dialog-title">Confirm Delete</h2> <p id="dialog-desc">This action cannot be undone.</p> <button>Cancel</button> <button>Delete</button> </div> ``` ```javascript // Focus management function openModal(modal) { // Store last focused element lastFocus = document.activeElement; // Move focus to modal modal.querySelector("h2").focus(); // Trap focus modal.addEventListener("keydown", trapFocus); } function closeModal(modal) { // Return focus lastFocus.focus(); } function trapFocus(e) { if (e.key === "Tab") { const focusable = modal.querySelectorAll( 'button, [href], input, select, textarea, [tabindex]:not([tabindex="-1"])', ); const first = focusable[0]; const last = focusable[focusable.length - 1]; if (e.shiftKey && document.activeElement === first) { last.focus(); e.preventDefault(); } else if (!e.shiftKey && document.activeElement === last) { first.focus(); e.preventDefault(); } } if (e.key === "Escape") { closeModal(modal); } } ``` ### 2. Live Regions ```html <!-- Status messages (polite) --> <div role="status" aria-live="polite" aria-atomic="true"> <!-- Content updates will be announced after current speech --> </div> <!-- Alerts (assertive) --> <div role="alert" aria-live="assertive"> <!-- Content updates interrupt current speech --> </div> <!-- Progress updates --> <div role="progressbar" aria-valuenow="75" aria-valuemin="0" aria-valuemax="100" aria-label="Upload progress" ></div> <!-- Log (additions only) --> <div role="log" aria-live="polite" aria-relevant="additions"> <!-- New messages announced, removals not --> </div> ``` ### 3. Tab Interface ```html <div role="tablist" aria-label="Product information"> <button role="tab" id="tab-1" aria-selected="true" aria-controls="panel-1"> Description </button> <button role="tab" id="tab-2" aria-selected="false" aria-controls="panel-2" tabindex="-1" > Reviews </button> </div> <div role="tabpanel" id="panel-1" aria-labelledby="tab-1"> Product description content... </div> <div role="tabpanel" id="panel-2" aria-labelledby="tab-2" hidden> Reviews content... </div> ``` ```javascript // Tab keyboard navigation tablist.addEventListener("keydown", (e) => { const tabs = [...tablist.querySelectorAll('[role="tab"]')]; const index = tabs.indexOf(document.activeElement); let newIndex; switch (e.key) { case "ArrowRight": newIndex = (index + 1) % tabs.length; break; case "ArrowLeft": newIndex = (index - 1 + tabs.length) % tabs.length; break; case "Home": newIndex = 0; break; case "End": newIndex = tabs.length - 1; break; default: return; } tabs[newIndex].focus(); activateTab(tabs[newIndex]); e.preventDefault(); }); ``` ## Debugging Tips ```javascript // Log what screen reader sees function logAccessibleName(element) { const computed = window.getComputedStyle(element); console.log({ role: element.getAttribute("role") || element.tagName, name: element.getAttribute("aria-label") || element.getAttribute("aria-labelledby") || element.textContent, state: { expanded: element.getAttribute("aria-expanded"), selected: element.getAttribute("aria-selected"), checked: element.getAttribute("aria-checked"), disabled: element.disabled, }, visible: computed.display !== "none" && computed.visibility !== "hidden", }); } ``` ## Best Practices ### Do's - **Test with actual screen readers** - Not just simulators - **Use semantic HTML first** - ARIA is supplemental - **Test in browse and focus modes** - Different experiences - **Verify focus management** - Especially for SPAs - **Test keyboard only first** - Foundation for SR testing ### Don'ts - **Don't assume one SR is enough** - Test multiple - **Don't ignore mobile** - Growing user base - **Don't test only happy path** - Test error states - **Don't skip dynamic content** - Most common issues - **Don't rely on visual testing** - Different experience ## Resources - [VoiceOver User Guide](https://support.apple.com/guide/voiceover/welcome/mac) - [NVDA User Guide](https://www.nvaccess.org/files/nvda/documentation/userGuide.html) - [JAWS Documentation](https://support.freedomscientific.com/Products/Blindness/JAWS) - [WebAIM Screen Reader Survey](https://webaim.org/projects/screenreadersurvey/)
πŸ‘1
πŸ‘οΈ1
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

data-quality-frameworks

Implement data quality validation with Great Expectations, dbt

data
⭐1
# Data Quality Frameworks Production patterns for implementing data quality with Great Expectations, dbt tests, and data contracts to ensure reliable data pipelines. ## When to Use This Skill - Implementing data quality checks in pipelines - Setting up Great Expectations validation - Building comprehensive dbt test suites - Establishing data contracts between teams - Monitoring data quality metrics - Automating data validation in CI/CD ## Core Concepts ### 1. Data Quality Dimensions | Dimension | Description | Example Check | | ---------------- | ------------------------ | -------------------------------------------------- | | **Completeness** | No missing values | `expect_column_values_to_not_be_null` | | **Uniqueness** | No duplicates | `expect_column_values_to_be_unique` | | **Validity** | Values in expected range | `expect_column_values_to_be_in_set` | | **Accuracy** | Data matches reality | Cross-reference validation | | **Consistency** | No contradictions | `expect_column_pair_values_A_to_be_greater_than_B` | | **Timeliness** | Data is recent | `expect_column_max_to_be_between` | ### 2. Testing Pyramid for Data ``` /\ / \ Integration Tests (cross-table) /────\ / \ Unit Tests (single column) /────────\ / \ Schema Tests (structure) /────────────\ ``` ## Quick Start ### Great Expectations Setup ```bash # Install pip install great_expectations # Initialize project great_expectations init # Create datasource great_expectations datasource new ``` ```python # great_expectations/checkpoints/daily_validation.yml import great_expectations as gx # Create context context = gx.get_context() # Create expectation suite suite = context.add_expectation_suite("orders_suite") # Add expectations suite.add_expectation( gx.expectations.ExpectColumnValuesToNotBeNull(column="order_id") ) suite.add_expectation( gx.expectations.ExpectColumnValuesToBeUnique(column="order_id") ) # Validate results = context.run_checkpoint(checkpoint_name="daily_orders") ``` ## Patterns ### Pattern 1: Great Expectations Suite ```python # expectations/orders_suite.py import great_expectations as gx from great_expectations.core import ExpectationSuite from great_expectations.core.expectation_configuration import ExpectationConfiguration def build_orders_suite() -> ExpectationSuite: """Build comprehensive orders expectation suite""" suite = ExpectationSuite(expectation_suite_name="orders_suite") # Schema expectations suite.add_expectation(ExpectationConfiguration( expectation_type="expect_table_columns_to_match_set", kwargs={ "column_set": ["order_id", "customer_id", "amount", "status", "created_at"], "exact_match": False # Allow additional columns } )) # Primary key suite.add_expectation(ExpectationConfiguration( expectation_type="expect_column_values_to_not_be_null", kwargs={"column": "order_id"} )) suite.add_expectation(ExpectationConfiguration( expectation_type="expect_column_values_to_be_unique", kwargs={"column": "order_id"} )) # Foreign key suite.add_expectation(ExpectationConfiguration( expectation_type="expect_column_values_to_not_be_null", kwargs={"column": "customer_id"} )) # Categorical values suite.add_expectation(ExpectationConfiguration( expectation_type="expect_column_values_to_be_in_set", kwargs={ "column": "status", "value_set": ["pending", "processing", "shipped", "delivered", "cancelled"] } )) # Numeric ranges suite.add_expectation(ExpectationConfiguration( expectation_type="expect_column_values_to_be_between", kwargs={ "column": "amount", "min_value": 0, "max_value": 100000, "strict_min": True # amount > 0 } )) # Date validity suite.add_expectation(ExpectationConfiguration( expectation_type="expect_column_values_to_be_dateutil_parseable", kwargs={"column": "created_at"} )) # Freshness - data should be recent suite.add_expectation(ExpectationConfiguration( expectation_type="expect_column_max_to_be_between", kwargs={ "column": "created_at", "min_value": {"$PARAMETER": "now - timedelta(days=1)"}, "max_value": {"$PARAMETER": "now"} } )) # Row count sanity suite.add_expectation(ExpectationConfiguration( expectation_type="expect_table_row_count_to_be_between", kwargs={ "min_value": 1000, # Expect at least 1000 rows "max_value": 10000000 } )) # Statistical expectations suite.add_expectation(ExpectationConfiguration( expectation_type="expect_column_mean_to_be_between", kwargs={ "column": "amount", "min_value": 50, "max_value": 500 } )) return suite ``` ### Pattern 2: Great Expectations Checkpoint ```yaml # great_expectations/checkpoints/orders_checkpoint.yml name: orders_checkpoint config_version: 1.0 class_name: Checkpoint run_name_template: "%Y%m%d-%H%M%S-orders-validation" validations: - batch_request: datasource_name: warehouse data_connector_name: default_inferred_data_connector_name data_asset_name: orders data_connector_query: index: -1 # Latest batch expectation_suite_name: orders_suite action_list: - name: store_validation_result action: class_name: StoreValidationResultAction - name: store_evaluation_parameters action: class_name: StoreEvaluationParametersAction - name: update_data_docs action: class_name: UpdateDataDocsAction # Slack notification on failure - name: send_slack_notification action: class_name: SlackNotificationAction slack_webhook: ${SLACK_WEBHOOK} notify_on: failure renderer: module_name: great_expectations.render.renderer.slack_renderer class_name: SlackRenderer ``` ```python # Run checkpoint import great_expectations as gx context = gx.get_context() result = context.run_checkpoint(checkpoint_name="orders_checkpoint") if not result.success: failed_expectations = [ r for r in result.run_results.values() if not r.success ] raise ValueError(f"Data quality check failed: {failed_expectations}") ``` ### Pattern 3: dbt Data Tests ```yaml # models/marts/core/_core__models.yml version: 2 models: - name: fct_orders description: Order fact table tests: # Table-level tests - dbt_utils.recency: datepart: day field: created_at interval: 1 - dbt_utils.at_least_one - dbt_utils.expression_is_true: expression: "total_amount >= 0" columns: - name: order_id description: Primary key tests: - unique - not_null - name: customer_id description: Foreign key to dim_customers tests: - not_null - relationships: to: ref('dim_customers') field: customer_id - name: order_status tests: - accepted_values: values: ["pending", "processing", "shipped", "delivered", "cancelled"] - name: total_amount tests: - not_null - dbt_utils.expression_is_true: expression: ">= 0" - name: created_at tests: - not_null - dbt_utils.expression_is_true: expression: "<= current_timestamp" - name: dim_customers columns: - name: customer_id tests: - unique - not_null - name: email tests: - unique - not_null # Custom regex test - dbt_utils.expression_is_true: expression: "email ~ '^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\\.[A-Za-z]{2,}$'" ``` ### Pattern 4: Custom dbt Tests ```sql -- tests/generic/test_row_count_in_range.sql {% test row_count_in_range(model, min_count, max_count) %} with row_count as ( select count(*) as cnt from {{ model }} ) select cnt from row_count where cnt < {{ min_count }} or cnt > {{ max_count }} {% endtest %} -- Usage in schema.yml: -- tests: -- - row_count_in_range: -- min_count: 1000 -- max_count: 10000000 ``` ```sql -- tests/generic/test_sequential_values.sql {% test sequential_values(model, column_name, interval=1) %} with lagged as ( select {{ column_name }}, lag({{ column_name }}) over (order by {{ column_name }}) as prev_value from {{ model }} ) select * from lagged where {{ column_name }} - prev_value != {{ interval }} and prev_value is not null {% endtest %} ``` ```sql -- tests/singular/assert_orders_customers_match.sql -- Singular test: specific business rule with orders_customers as ( select distinct customer_id from {{ ref('fct_orders') }} ), dim_customers as ( select customer_id from {{ ref('dim_customers') }} ), orphaned_orders as ( select o.customer_id from orders_customers o left join dim_customers c using (customer_id) where c.customer_id is null ) select * from orphaned_orders -- Test passes if this returns 0 rows ``` ### Pattern 5: Data Contracts ```yaml # contracts/orders_contract.yaml apiVersion: datacontract.com/v1.0.0 kind: DataContract metadata: name: orders version: 1.0.0 owner: data-platform-team contact: data-team@company.com info: title: Orders Data Contract description: Contract for order event data from the ecommerce platform purpose: Analytics, reporting, and ML features servers: production: type: snowflake account: company.us-east-1 database: ANALYTICS schema: CORE terms: usage: Internal analytics only limitations: PII must not be exposed in downstream marts billing: Charged per query TB scanned schema: type: object properties: order_id: type: string format: uuid description: Unique order identifier required: true unique: true pii: false customer_id: type: string format: uuid description: Customer identifier required: true pii: true piiClassification: indirect total_amount: type: number minimum: 0 maximum: 100000 description: Order total in USD created_at: type: string format: date-time description: Order creation timestamp required: true status: type: string enum: [pending, processing, shipped, delivered, cancelled] description: Current order status quality: type: SodaCL specification: checks for orders: - row_count > 0 - missing_count(order_id) = 0 - duplicate_count(order_id) = 0 - invalid_count(status) = 0: valid values: [pending, processing, shipped, delivered, cancelled] - freshness(created_at) < 24h sla: availability: 99.9% freshness: 1 hour latency: 5 minutes ``` ### Pattern 6: Automated Quality Pipeline ```python # quality_pipeline.py from dataclasses import dataclass from typing import List, Dict, Any import great_expectations as gx from datetime import datetime @dataclass class QualityResult: table: str passed: bool total_expectations: int failed_expectations: int details: List[Dict[str, Any]] timestamp: datetime class DataQualityPipeline: """Orchestrate data quality checks across tables""" def __init__(self, context: gx.DataContext): self.context = context self.results: List[QualityResult] = [] def validate_table(self, table: str, suite: str) -> QualityResult: """Validate a single table against expectation suite""" checkpoint_config = { "name": f"{table}_validation", "config_version": 1.0, "class_name": "Checkpoint", "validations": [{ "batch_request": { "datasource_name": "warehouse", "data_asset_name": table, }, "expectation_suite_name": suite, }], } result = self.context.run_checkpoint(**checkpoint_config) # Parse results validation_result = list(result.run_results.values())[0] results = validation_result.results failed = [r for r in results if not r.success] return QualityResult( table=table, passed=result.success, total_expectations=len(results), failed_expectations=len(failed), details=[{ "expectation": r.expectation_config.expectation_type, "success": r.success, "observed_value": r.result.get("observed_value"), } for r in results], timestamp=datetime.now() ) def run_all(self, tables: Dict[str, str]) -> Dict[str, QualityResult]: """Run validation for all tables""" results = {} for table, suite in tables.items(): print(f"Validating {table}...") results[table] = self.validate_table(table, suite) return results def generate_report(self, results: Dict[str, QualityResult]) -> str: """Generate quality report""" report = ["# Data Quality Report", f"Generated: {datetime.now()}", ""] total_passed = sum(1 for r in results.values() if r.passed) total_tables = len(results) report.append(f"## Summary: {total_passed}/{total_tables} tables passed") report.append("") for table, result in results.items(): status = "βœ…" if result.passed else "❌" report.append(f"### {status} {table}") report.append(f"- Expectations: {result.total_expectations}") report.append(f"- Failed: {result.failed_expectations}") if not result.passed: report.append("- Failed checks:") for detail in result.details: if not detail["success"]: report.append(f" - {detail['expectation']}: {detail['observed_value']}") report.append("") return "\n".join(report) # Usage context = gx.get_context() pipeline = DataQualityPipeline(context) tables_to_validate = { "orders": "orders_suite", "customers": "customers_suite", "products": "products_suite", } results = pipeline.run_all(tables_to_validate) report = pipeline.generate_report(results) # Fail pipeline if any table failed if not all(r.passed for r in results.values()): print(report) raise ValueError("Data quality checks failed!") ``` ## Best Practices ### Do's - **Test early** - Validate source data before transformations - **Test incrementally** - Add tests as you find issues - **Document expectations** - Clear descriptions for each test - **Alert on failures** - Integrate with monitoring - **Version contracts** - Track schema changes ### Don'ts - **Don't test everything** - Focus on critical columns - **Don't ignore warnings** - They often precede failures - **Don't skip freshness** - Stale data is bad data - **Don't hardcode thresholds** - Use dynamic baselines - **Don't test in isolation** - Test relationships too ## Resources - [Great Expectations Documentation](https://docs.greatexpectations.io/) - [dbt Testing Documentation](https://docs.getdbt.com/docs/build/tests) - [Data Contract Specification](https://datacontract.com/) - [Soda Core](https://docs.soda.io/soda-core/overview.html)
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

bazel-build-optimization

Optimize Bazel builds for large-scale monorepos. Use when

coding
⭐1
# Bazel Build Optimization Production patterns for Bazel in large-scale monorepos. ## When to Use This Skill - Setting up Bazel for monorepos - Configuring remote caching/execution - Optimizing build times - Writing custom Bazel rules - Debugging build issues - Migrating to Bazel ## Core Concepts ### 1. Bazel Architecture ``` workspace/ β”œβ”€β”€ WORKSPACE.bazel # External dependencies β”œβ”€β”€ .bazelrc # Build configurations β”œβ”€β”€ .bazelversion # Bazel version β”œβ”€β”€ BUILD.bazel # Root build file β”œβ”€β”€ apps/ β”‚ └── web/ β”‚ └── BUILD.bazel β”œβ”€β”€ libs/ β”‚ └── utils/ β”‚ └── BUILD.bazel └── tools/ └── bazel/ └── rules/ ``` ### 2. Key Concepts | Concept | Description | | ----------- | -------------------------------------- | | **Target** | Buildable unit (library, binary, test) | | **Package** | Directory with BUILD file | | **Label** | Target identifier `//path/to:target` | | **Rule** | Defines how to build a target | | **Aspect** | Cross-cutting build behavior | ## Templates ### Template 1: WORKSPACE Configuration ```python # WORKSPACE.bazel workspace(name = "myproject") load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") # Rules for JavaScript/TypeScript http_archive( name = "aspect_rules_js", sha256 = "...", strip_prefix = "rules_js-1.34.0", url = "https://github.com/aspect-build/rules_js/releases/download/v1.34.0/rules_js-v1.34.0.tar.gz", ) load("@aspect_rules_js//js:repositories.bzl", "rules_js_dependencies") rules_js_dependencies() load("@rules_nodejs//nodejs:repositories.bzl", "nodejs_register_toolchains") nodejs_register_toolchains( name = "nodejs", node_version = "20.9.0", ) load("@aspect_rules_js//npm:repositories.bzl", "npm_translate_lock") npm_translate_lock( name = "npm", pnpm_lock = "//:pnpm-lock.yaml", verify_node_modules_ignored = "//:.bazelignore", ) load("@npm//:repositories.bzl", "npm_repositories") npm_repositories() # Rules for Python http_archive( name = "rules_python", sha256 = "...", strip_prefix = "rules_python-0.27.0", url = "https://github.com/bazelbuild/rules_python/releases/download/0.27.0/rules_python-0.27.0.tar.gz", ) load("@rules_python//python:repositories.bzl", "py_repositories") py_repositories() ``` ### Template 2: .bazelrc Configuration ```bash # .bazelrc # Build settings build --enable_platform_specific_config build --incompatible_enable_cc_toolchain_resolution build --experimental_strict_conflict_checks # Performance build --jobs=auto build --local_cpu_resources=HOST_CPUS*.75 build --local_ram_resources=HOST_RAM*.75 # Caching build --disk_cache=~/.cache/bazel-disk build --repository_cache=~/.cache/bazel-repo # Remote caching (optional) build:remote-cache --remote_cache=grpcs://cache.example.com build:remote-cache --remote_upload_local_results=true build:remote-cache --remote_timeout=3600 # Remote execution (optional) build:remote-exec --remote_executor=grpcs://remote.example.com build:remote-exec --remote_instance_name=projects/myproject/instances/default build:remote-exec --jobs=500 # Platform configurations build:linux --platforms=//platforms:linux_x86_64 build:macos --platforms=//platforms:macos_arm64 # CI configuration build:ci --config=remote-cache build:ci --build_metadata=ROLE=CI build:ci --bes_results_url=https://results.example.com/invocation/ build:ci --bes_backend=grpcs://bes.example.com # Test settings test --test_output=errors test --test_summary=detailed # Coverage coverage --combined_report=lcov coverage --instrumentation_filter="//..." # Convenience aliases build:opt --compilation_mode=opt build:dbg --compilation_mode=dbg # Import user settings try-import %workspace%/user.bazelrc ``` ### Template 3: TypeScript Library BUILD ```python # libs/utils/BUILD.bazel load("@aspect_rules_ts//ts:defs.bzl", "ts_project") load("@aspect_rules_js//js:defs.bzl", "js_library") load("@npm//:defs.bzl", "npm_link_all_packages") npm_link_all_packages(name = "node_modules") ts_project( name = "utils_ts", srcs = glob(["src/**/*.ts"]), declaration = True, source_map = True, tsconfig = "//:tsconfig.json", deps = [ ":node_modules/@types/node", ], ) js_library( name = "utils", srcs = [":utils_ts"], visibility = ["//visibility:public"], ) # Tests load("@aspect_rules_jest//jest:defs.bzl", "jest_test") jest_test( name = "utils_test", config = "//:jest.config.js", data = [ ":utils", "//:node_modules/jest", ], node_modules = "//:node_modules", ) ``` ### Template 4: Python Library BUILD ```python # libs/ml/BUILD.bazel load("@rules_python//python:defs.bzl", "py_library", "py_test", "py_binary") load("@pip//:requirements.bzl", "requirement") py_library( name = "ml", srcs = glob(["src/**/*.py"]), deps = [ requirement("numpy"), requirement("pandas"), requirement("scikit-learn"), "//libs/utils:utils_py", ], visibility = ["//visibility:public"], ) py_test( name = "ml_test", srcs = glob(["tests/**/*.py"]), deps = [ ":ml", requirement("pytest"), ], size = "medium", timeout = "moderate", ) py_binary( name = "train", srcs = ["train.py"], deps = [":ml"], data = ["//data:training_data"], ) ``` ### Template 5: Custom Rule for Docker ```python # tools/bazel/rules/docker.bzl def _docker_image_impl(ctx): dockerfile = ctx.file.dockerfile base_image = ctx.attr.base_image layers = ctx.files.layers # Build the image output = ctx.actions.declare_file(ctx.attr.name + ".tar") args = ctx.actions.args() args.add("--dockerfile", dockerfile) args.add("--output", output) args.add("--base", base_image) args.add_all("--layer", layers) ctx.actions.run( inputs = [dockerfile] + layers, outputs = [output], executable = ctx.executable._builder, arguments = [args], mnemonic = "DockerBuild", progress_message = "Building Docker image %s" % ctx.label, ) return [DefaultInfo(files = depset([output]))] docker_image = rule( implementation = _docker_image_impl, attrs = { "dockerfile": attr.label( allow_single_file = [".dockerfile", "Dockerfile"], mandatory = True, ), "base_image": attr.string(mandatory = True), "layers": attr.label_list(allow_files = True), "_builder": attr.label( default = "//tools/docker:builder", executable = True, cfg = "exec", ), }, ) ``` ### Template 6: Query and Dependency Analysis ```bash # Find all dependencies of a target bazel query "deps(//apps/web:web)" # Find reverse dependencies (what depends on this) bazel query "rdeps(//..., //libs/utils:utils)" # Find all targets in a package bazel query "//libs/..." # Find changed targets since commit bazel query "rdeps(//..., set($(git diff --name-only HEAD~1 | sed 's/.*/"&"/' | tr '\n' ' ')))" # Generate dependency graph bazel query "deps(//apps/web:web)" --output=graph | dot -Tpng > deps.png # Find all test targets bazel query "kind('.*_test', //...)" # Find targets with specific tag bazel query "attr(tags, 'integration', //...)" # Compute build graph size bazel query "deps(//...)" --output=package | wc -l ``` ### Template 7: Remote Execution Setup ```python # platforms/BUILD.bazel platform( name = "linux_x86_64", constraint_values = [ "@platforms//os:linux", "@platforms//cpu:x86_64", ], exec_properties = { "container-image": "docker://gcr.io/myproject/bazel-worker:latest", "OSFamily": "Linux", }, ) platform( name = "remote_linux", parents = [":linux_x86_64"], exec_properties = { "Pool": "default", "dockerNetwork": "standard", }, ) # toolchains/BUILD.bazel toolchain( name = "cc_toolchain_linux", exec_compatible_with = [ "@platforms//os:linux", "@platforms//cpu:x86_64", ], target_compatible_with = [ "@platforms//os:linux", "@platforms//cpu:x86_64", ], toolchain = "@remotejdk11_linux//:jdk", toolchain_type = "@bazel_tools//tools/jdk:runtime_toolchain_type", ) ``` ## Performance Optimization ```bash # Profile build bazel build //... --profile=profile.json bazel analyze-profile profile.json # Identify slow actions bazel build //... --execution_log_json_file=exec_log.json # Memory profiling bazel build //... --memory_profile=memory.json # Skip analysis cache bazel build //... --notrack_incremental_state ``` ## Best Practices ### Do's - **Use fine-grained targets** - Better caching - **Pin dependencies** - Reproducible builds - **Enable remote caching** - Share build artifacts - **Use visibility wisely** - Enforce architecture - **Write BUILD files per directory** - Standard convention ### Don'ts - **Don't use glob for deps** - Explicit is better - **Don't commit bazel-\* dirs** - Add to .gitignore - **Don't skip WORKSPACE setup** - Foundation of build - **Don't ignore build warnings** - Technical debt ## Resources - [Bazel Documentation](https://bazel.build/docs) - [Bazel Remote Execution](https://bazel.build/docs/remote-execution) - [rules_js](https://github.com/aspect-build/rules_js)
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

e2e-testing-patterns

Master end-to-end testing with Playwright and Cypress to build

coding
⭐1
# E2E Testing Patterns Build reliable, fast, and maintainable end-to-end test suites that provide confidence to ship code quickly and catch regressions before users do. ## When to Use This Skill - Implementing end-to-end test automation - Debugging flaky or unreliable tests - Testing critical user workflows - Setting up CI/CD test pipelines - Testing across multiple browsers - Validating accessibility requirements - Testing responsive designs - Establishing E2E testing standards ## Core Concepts ### 1. E2E Testing Fundamentals **What to Test with E2E:** - Critical user journeys (login, checkout, signup) - Complex interactions (drag-and-drop, multi-step forms) - Cross-browser compatibility - Real API integration - Authentication flows **What NOT to Test with E2E:** - Unit-level logic (use unit tests) - API contracts (use integration tests) - Edge cases (too slow) - Internal implementation details ### 2. Test Philosophy **The Testing Pyramid:** ``` /\ /E2E\ ← Few, focused on critical paths /─────\ /Integr\ ← More, test component interactions /────────\ /Unit Tests\ ← Many, fast, isolated /────────────\ ``` **Best Practices:** - Test user behavior, not implementation - Keep tests independent - Make tests deterministic - Optimize for speed - Use data-testid, not CSS selectors ## Playwright Patterns ### Setup and Configuration ```typescript // playwright.config.ts import { defineConfig, devices } from "@playwright/test"; export default defineConfig({ testDir: "./e2e", timeout: 30000, expect: { timeout: 5000, }, fullyParallel: true, forbidOnly: !!process.env.CI, retries: process.env.CI ? 2 : 0, workers: process.env.CI ? 1 : undefined, reporter: [["html"], ["junit", { outputFile: "results.xml" }]], use: { baseURL: "http://localhost:3000", trace: "on-first-retry", screenshot: "only-on-failure", video: "retain-on-failure", }, projects: [ { name: "chromium", use: { ...devices["Desktop Chrome"] } }, { name: "firefox", use: { ...devices["Desktop Firefox"] } }, { name: "webkit", use: { ...devices["Desktop Safari"] } }, { name: "mobile", use: { ...devices["iPhone 13"] } }, ], }); ``` ### Pattern 1: Page Object Model ```typescript // pages/LoginPage.ts import { Page, Locator } from "@playwright/test"; export class LoginPage { readonly page: Page; readonly emailInput: Locator; readonly passwordInput: Locator; readonly loginButton: Locator; readonly errorMessage: Locator; constructor(page: Page) { this.page = page; this.emailInput = page.getByLabel("Email"); this.passwordInput = page.getByLabel("Password"); this.loginButton = page.getByRole("button", { name: "Login" }); this.errorMessage = page.getByRole("alert"); } async goto() { await this.page.goto("/login"); } async login(email: string, password: string) { await this.emailInput.fill(email); await this.passwordInput.fill(password); await this.loginButton.click(); } async getErrorMessage(): Promise<string> { return (await this.errorMessage.textContent()) ?? ""; } } // Test using Page Object import { test, expect } from "@playwright/test"; import { LoginPage } from "./pages/LoginPage"; test("successful login", async ({ page }) => { const loginPage = new LoginPage(page); await loginPage.goto(); await loginPage.login("user@example.com", "password123"); await expect(page).toHaveURL("/dashboard"); await expect(page.getByRole("heading", { name: "Dashboard" })).toBeVisible(); }); test("failed login shows error", async ({ page }) => { const loginPage = new LoginPage(page); await loginPage.goto(); await loginPage.login("invalid@example.com", "wrong"); const error = await loginPage.getErrorMessage(); expect(error).toContain("Invalid credentials"); }); ``` ### Pattern 2: Fixtures for Test Data ```typescript // fixtures/test-data.ts import { test as base } from "@playwright/test"; type TestData = { testUser: { email: string; password: string; name: string; }; adminUser: { email: string; password: string; }; }; export const test = base.extend<TestData>({ testUser: async ({}, use) => { const user = { email: `test-${Date.now()}@example.com`, password: "Test123!@#", name: "Test User", }; // Setup: Create user in database await createTestUser(user); await use(user); // Teardown: Clean up user await deleteTestUser(user.email); }, adminUser: async ({}, use) => { await use({ email: "admin@example.com", password: process.env.ADMIN_PASSWORD!, }); }, }); // Usage in tests import { test } from "./fixtures/test-data"; test("user can update profile", async ({ page, testUser }) => { await page.goto("/login"); await page.getByLabel("Email").fill(testUser.email); await page.getByLabel("Password").fill(testUser.password); await page.getByRole("button", { name: "Login" }).click(); await page.goto("/profile"); await page.getByLabel("Name").fill("Updated Name"); await page.getByRole("button", { name: "Save" }).click(); await expect(page.getByText("Profile updated")).toBeVisible(); }); ``` ### Pattern 3: Waiting Strategies ```typescript // ❌ Bad: Fixed timeouts await page.waitForTimeout(3000); // Flaky! // βœ… Good: Wait for specific conditions await page.waitForLoadState("networkidle"); await page.waitForURL("/dashboard"); await page.waitForSelector('[data-testid="user-profile"]'); // βœ… Better: Auto-waiting with assertions await expect(page.getByText("Welcome")).toBeVisible(); await expect(page.getByRole("button", { name: "Submit" })).toBeEnabled(); // Wait for API response const responsePromise = page.waitForResponse( (response) => response.url().includes("/api/users") && response.status() === 200, ); await page.getByRole("button", { name: "Load Users" }).click(); const response = await responsePromise; const data = await response.json(); expect(data.users).toHaveLength(10); // Wait for multiple conditions await Promise.all([ page.waitForURL("/success"), page.waitForLoadState("networkidle"), expect(page.getByText("Payment successful")).toBeVisible(), ]); ``` ### Pattern 4: Network Mocking and Interception ```typescript // Mock API responses test("displays error when API fails", async ({ page }) => { await page.route("**/api/users", (route) => { route.fulfill({ status: 500, contentType: "application/json", body: JSON.stringify({ error: "Internal Server Error" }), }); }); await page.goto("/users"); await expect(page.getByText("Failed to load users")).toBeVisible(); }); // Intercept and modify requests test("can modify API request", async ({ page }) => { await page.route("**/api/users", async (route) => { const request = route.request(); const postData = JSON.parse(request.postData() || "{}"); // Modify request postData.role = "admin"; await route.continue({ postData: JSON.stringify(postData), }); }); // Test continues... }); // Mock third-party services test("payment flow with mocked Stripe", async ({ page }) => { await page.route("**/api/stripe/**", (route) => { route.fulfill({ status: 200, body: JSON.stringify({ id: "mock_payment_id", status: "succeeded", }), }); }); // Test payment flow with mocked response }); ``` ## Cypress Patterns ### Setup and Configuration ```typescript // cypress.config.ts import { defineConfig } from "cypress"; export default defineConfig({ e2e: { baseUrl: "http://localhost:3000", viewportWidth: 1280, viewportHeight: 720, video: false, screenshotOnRunFailure: true, defaultCommandTimeout: 10000, requestTimeout: 10000, setupNodeEvents(on, config) { // Implement node event listeners }, }, }); ``` ### Pattern 1: Custom Commands ```typescript // cypress/support/commands.ts declare global { namespace Cypress { interface Chainable { login(email: string, password: string): Chainable<void>; createUser(userData: UserData): Chainable<User>; dataCy(value: string): Chainable<JQuery<HTMLElement>>; } } } Cypress.Commands.add("login", (email: string, password: string) => { cy.visit("/login"); cy.get('[data-testid="email"]').type(email); cy.get('[data-testid="password"]').type(password); cy.get('[data-testid="login-button"]').click(); cy.url().should("include", "/dashboard"); }); Cypress.Commands.add("createUser", (userData: UserData) => { return cy.request("POST", "/api/users", userData).its("body"); }); Cypress.Commands.add("dataCy", (value: string) => { return cy.get(`[data-cy="${value}"]`); }); // Usage cy.login("user@example.com", "password"); cy.dataCy("submit-button").click(); ``` ### Pattern 2: Cypress Intercept ```typescript // Mock API calls cy.intercept("GET", "/api/users", { statusCode: 200, body: [ { id: 1, name: "John" }, { id: 2, name: "Jane" }, ], }).as("getUsers"); cy.visit("/users"); cy.wait("@getUsers"); cy.get('[data-testid="user-list"]').children().should("have.length", 2); // Modify responses cy.intercept("GET", "/api/users", (req) => { req.reply((res) => { // Modify response res.body.users = res.body.users.slice(0, 5); res.send(); }); }); // Simulate slow network cy.intercept("GET", "/api/data", (req) => { req.reply((res) => { res.delay(3000); // 3 second delay res.send(); }); }); ``` ## Advanced Patterns ### Pattern 1: Visual Regression Testing ```typescript // With Playwright import { test, expect } from "@playwright/test"; test("homepage looks correct", async ({ page }) => { await page.goto("/"); await expect(page).toHaveScreenshot("homepage.png", { fullPage: true, maxDiffPixels: 100, }); }); test("button in all states", async ({ page }) => { await page.goto("/components"); const button = page.getByRole("button", { name: "Submit" }); // Default state await expect(button).toHaveScreenshot("button-default.png"); // Hover state await button.hover(); await expect(button).toHaveScreenshot("button-hover.png"); // Disabled state await button.evaluate((el) => el.setAttribute("disabled", "true")); await expect(button).toHaveScreenshot("button-disabled.png"); }); ``` ### Pattern 2: Parallel Testing with Sharding ```typescript // playwright.config.ts export default defineConfig({ projects: [ { name: "shard-1", use: { ...devices["Desktop Chrome"] }, grepInvert: /@slow/, shard: { current: 1, total: 4 }, }, { name: "shard-2", use: { ...devices["Desktop Chrome"] }, shard: { current: 2, total: 4 }, }, // ... more shards ], }); // Run in CI // npx playwright test --shard=1/4 // npx playwright test --shard=2/4 ``` ### Pattern 3: Accessibility Testing ```typescript // Install: npm install @axe-core/playwright import { test, expect } from "@playwright/test"; import AxeBuilder from "@axe-core/playwright"; test("page should not have accessibility violations", async ({ page }) => { await page.goto("/"); const accessibilityScanResults = await new AxeBuilder({ page }) .exclude("#third-party-widget") .analyze(); expect(accessibilityScanResults.violations).toEqual([]); }); test("form is accessible", async ({ page }) => { await page.goto("/signup"); const results = await new AxeBuilder({ page }).include("form").analyze(); expect(results.violations).toEqual([]); }); ``` ## Best Practices 1. **Use Data Attributes**: `data-testid` or `data-cy` for stable selectors 2. **Avoid Brittle Selectors**: Don't rely on CSS classes or DOM structure 3. **Test User Behavior**: Click, type, see - not implementation details 4. **Keep Tests Independent**: Each test should run in isolation 5. **Clean Up Test Data**: Create and destroy test data in each test 6. **Use Page Objects**: Encapsulate page logic 7. **Meaningful Assertions**: Check actual user-visible behavior 8. **Optimize for Speed**: Mock when possible, parallel execution ```typescript // ❌ Bad selectors cy.get(".btn.btn-primary.submit-button").click(); cy.get("div > form > div:nth-child(2) > input").type("text"); // βœ… Good selectors cy.getByRole("button", { name: "Submit" }).click(); cy.getByLabel("Email address").type("user@example.com"); cy.get('[data-testid="email-input"]').type("user@example.com"); ``` ## Common Pitfalls - **Flaky Tests**: Use proper waits, not fixed timeouts - **Slow Tests**: Mock external APIs, use parallel execution - **Over-Testing**: Don't test every edge case with E2E - **Coupled Tests**: Tests should not depend on each other - **Poor Selectors**: Avoid CSS classes and nth-child - **No Cleanup**: Clean up test data after each test - **Testing Implementation**: Test user behavior, not internals ## Debugging Failing Tests ```typescript // Playwright debugging // 1. Run in headed mode npx playwright test --headed // 2. Run in debug mode npx playwright test --debug // 3. Use trace viewer await page.screenshot({ path: 'screenshot.png' }); await page.video()?.saveAs('video.webm'); // 4. Add test.step for better reporting test('checkout flow', async ({ page }) => { await test.step('Add item to cart', async () => { await page.goto('/products'); await page.getByRole('button', { name: 'Add to Cart' }).click(); }); await test.step('Proceed to checkout', async () => { await page.goto('/cart'); await page.getByRole('button', { name: 'Checkout' }).click(); }); }); // 5. Inspect page state await page.pause(); // Pauses execution, opens inspector ``` ## Resources - **references/playwright-best-practices.md**: Playwright-specific patterns - **references/cypress-best-practices.md**: Cypress-specific patterns - **references/flaky-test-debugging.md**: Debugging unreliable tests - **assets/e2e-testing-checklist.md**: What to test with E2E - **assets/selector-strategies.md**: Finding reliable selectors - **scripts/test-analyzer.ts**: Analyze test flakiness and duration
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

monorepo-management

Master monorepo management with Turborepo, Nx, and pnpm workspaces

coding
⭐1
# Monorepo Management Build efficient, scalable monorepos that enable code sharing, consistent tooling, and atomic changes across multiple packages and applications. ## When to Use This Skill - Setting up new monorepo projects - Migrating from multi-repo to monorepo - Optimizing build and test performance - Managing shared dependencies - Implementing code sharing strategies - Setting up CI/CD for monorepos - Versioning and publishing packages - Debugging monorepo-specific issues ## Core Concepts ### 1. Why Monorepos? **Advantages:** - Shared code and dependencies - Atomic commits across projects - Consistent tooling and standards - Easier refactoring - Simplified dependency management - Better code visibility **Challenges:** - Build performance at scale - CI/CD complexity - Access control - Large Git repository ### 2. Monorepo Tools **Package Managers:** - pnpm workspaces (recommended) - npm workspaces - Yarn workspaces **Build Systems:** - Turborepo (recommended for most) - Nx (feature-rich, complex) - Lerna (older, maintenance mode) ## Turborepo Setup ### Initial Setup ```bash # Create new monorepo npx create-turbo@latest my-monorepo cd my-monorepo # Structure: # apps/ # web/ - Next.js app # docs/ - Documentation site # packages/ # ui/ - Shared UI components # config/ - Shared configurations # tsconfig/ - Shared TypeScript configs # turbo.json - Turborepo configuration # package.json - Root package.json ``` ### Configuration ```json // turbo.json { "$schema": "https://turbo.build/schema.json", "globalDependencies": ["**/.env.*local"], "pipeline": { "build": { "dependsOn": ["^build"], "outputs": ["dist/**", ".next/**", "!.next/cache/**"] }, "test": { "dependsOn": ["build"], "outputs": ["coverage/**"] }, "lint": { "outputs": [] }, "dev": { "cache": false, "persistent": true }, "type-check": { "dependsOn": ["^build"], "outputs": [] } } } ``` ```json // package.json (root) { "name": "my-monorepo", "private": true, "workspaces": ["apps/*", "packages/*"], "scripts": { "build": "turbo run build", "dev": "turbo run dev", "test": "turbo run test", "lint": "turbo run lint", "format": "prettier --write \"**/*.{ts,tsx,md}\"", "clean": "turbo run clean && rm -rf node_modules" }, "devDependencies": { "turbo": "^1.10.0", "prettier": "^3.0.0", "typescript": "^5.0.0" }, "packageManager": "pnpm@8.0.0" } ``` ### Package Structure ```json // packages/ui/package.json { "name": "@repo/ui", "version": "0.0.0", "private": true, "main": "./dist/index.js", "types": "./dist/index.d.ts", "exports": { ".": { "import": "./dist/index.js", "types": "./dist/index.d.ts" }, "./button": { "import": "./dist/button.js", "types": "./dist/button.d.ts" } }, "scripts": { "build": "tsup src/index.ts --format esm,cjs --dts", "dev": "tsup src/index.ts --format esm,cjs --dts --watch", "lint": "eslint src/", "type-check": "tsc --noEmit" }, "devDependencies": { "@repo/tsconfig": "workspace:*", "tsup": "^7.0.0", "typescript": "^5.0.0" }, "dependencies": { "react": "^18.2.0" } } ``` ## pnpm Workspaces ### Setup ```yaml # pnpm-workspace.yaml packages: - "apps/*" - "packages/*" - "tools/*" ``` ```json // .npmrc # Hoist shared dependencies shamefully-hoist=true # Strict peer dependencies auto-install-peers=true strict-peer-dependencies=true # Performance store-dir=~/.pnpm-store ``` ### Dependency Management ```bash # Install dependency in specific package pnpm add react --filter @repo/ui pnpm add -D typescript --filter @repo/ui # Install workspace dependency pnpm add @repo/ui --filter web # Install in all packages pnpm add -D eslint -w # Update all dependencies pnpm update -r # Remove dependency pnpm remove react --filter @repo/ui ``` ### Scripts ```bash # Run script in specific package pnpm --filter web dev pnpm --filter @repo/ui build # Run in all packages pnpm -r build pnpm -r test # Run in parallel pnpm -r --parallel dev # Filter by pattern pnpm --filter "@repo/*" build pnpm --filter "...web" build # Build web and dependencies ``` ## Nx Monorepo ### Setup ```bash # Create Nx monorepo npx create-nx-workspace@latest my-org # Generate applications nx generate @nx/react:app my-app nx generate @nx/next:app my-next-app # Generate libraries nx generate @nx/react:lib ui-components nx generate @nx/js:lib utils ``` ### Configuration ```json // nx.json { "extends": "nx/presets/npm.json", "$schema": "./node_modules/nx/schemas/nx-schema.json", "targetDefaults": { "build": { "dependsOn": ["^build"], "inputs": ["production", "^production"], "cache": true }, "test": { "inputs": ["default", "^production", "{workspaceRoot}/jest.preset.js"], "cache": true }, "lint": { "inputs": ["default", "{workspaceRoot}/.eslintrc.json"], "cache": true } }, "namedInputs": { "default": ["{projectRoot}/**/*", "sharedGlobals"], "production": [ "default", "!{projectRoot}/**/?(*.)+(spec|test).[jt]s?(x)?(.snap)", "!{projectRoot}/tsconfig.spec.json" ], "sharedGlobals": [] } } ``` ### Running Tasks ```bash # Run task for specific project nx build my-app nx test ui-components nx lint utils # Run for affected projects nx affected:build nx affected:test --base=main # Visualize dependencies nx graph # Run in parallel nx run-many --target=build --all --parallel=3 ``` ## Shared Configurations ### TypeScript Configuration ```json // packages/tsconfig/base.json { "compilerOptions": { "strict": true, "esModuleInterop": true, "skipLibCheck": true, "forceConsistentCasingInFileNames": true, "module": "ESNext", "moduleResolution": "bundler", "resolveJsonModule": true, "isolatedModules": true, "incremental": true, "declaration": true }, "exclude": ["node_modules"] } // packages/tsconfig/react.json { "extends": "./base.json", "compilerOptions": { "jsx": "react-jsx", "lib": ["ES2022", "DOM", "DOM.Iterable"] } } // apps/web/tsconfig.json { "extends": "@repo/tsconfig/react.json", "compilerOptions": { "outDir": "dist", "rootDir": "src" }, "include": ["src"], "exclude": ["node_modules", "dist"] } ``` ### ESLint Configuration ```javascript // packages/config/eslint-preset.js module.exports = { extends: [ "eslint:recommended", "plugin:@typescript-eslint/recommended", "plugin:react/recommended", "plugin:react-hooks/recommended", "prettier", ], plugins: ["@typescript-eslint", "react", "react-hooks"], parser: "@typescript-eslint/parser", parserOptions: { ecmaVersion: 2022, sourceType: "module", ecmaFeatures: { jsx: true, }, }, settings: { react: { version: "detect", }, }, rules: { "@typescript-eslint/no-unused-vars": "error", "react/react-in-jsx-scope": "off", }, }; // apps/web/.eslintrc.js module.exports = { extends: ["@repo/config/eslint-preset"], rules: { // App-specific rules }, }; ``` ## Code Sharing Patterns ### Pattern 1: Shared UI Components ```typescript // packages/ui/src/button.tsx import * as React from 'react'; export interface ButtonProps { variant?: 'primary' | 'secondary'; children: React.ReactNode; onClick?: () => void; } export function Button({ variant = 'primary', children, onClick }: ButtonProps) { return ( <button className={`btn btn-${variant}`} onClick={onClick} > {children} </button> ); } // packages/ui/src/index.ts export { Button, type ButtonProps } from './button'; export { Input, type InputProps } from './input'; // apps/web/src/app.tsx import { Button } from '@repo/ui'; export function App() { return <Button variant="primary">Click me</Button>; } ``` ### Pattern 2: Shared Utilities ```typescript // packages/utils/src/string.ts export function capitalize(str: string): string { return str.charAt(0).toUpperCase() + str.slice(1); } export function truncate(str: string, length: number): string { return str.length > length ? str.slice(0, length) + "..." : str; } // packages/utils/src/index.ts export * from "./string"; export * from "./array"; export * from "./date"; // Usage in apps import { capitalize, truncate } from "@repo/utils"; ``` ### Pattern 3: Shared Types ```typescript // packages/types/src/user.ts export interface User { id: string; email: string; name: string; role: "admin" | "user"; } export interface CreateUserInput { email: string; name: string; password: string; } // Used in both frontend and backend import type { User, CreateUserInput } from "@repo/types"; ``` ## Build Optimization ### Turborepo Caching ```json // turbo.json { "pipeline": { "build": { // Build depends on dependencies being built first "dependsOn": ["^build"], // Cache these outputs "outputs": ["dist/**", ".next/**"], // Cache based on these inputs (default: all files) "inputs": ["src/**/*.tsx", "src/**/*.ts", "package.json"] }, "test": { // Run tests in parallel, don't depend on build "cache": true, "outputs": ["coverage/**"] } } } ``` ### Remote Caching ```bash # Turborepo Remote Cache (Vercel) npx turbo login npx turbo link # Custom remote cache # turbo.json { "remoteCache": { "signature": true, "enabled": true } } ``` ## CI/CD for Monorepos ### GitHub Actions ```yaml # .github/workflows/ci.yml name: CI on: push: branches: [main] pull_request: branches: [main] jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 with: fetch-depth: 0 # For Nx affected commands - uses: pnpm/action-setup@v2 with: version: 8 - uses: actions/setup-node@v3 with: node-version: 18 cache: "pnpm" - name: Install dependencies run: pnpm install --frozen-lockfile - name: Build run: pnpm turbo run build - name: Test run: pnpm turbo run test - name: Lint run: pnpm turbo run lint - name: Type check run: pnpm turbo run type-check ``` ### Deploy Affected Only ```yaml # Deploy only changed apps - name: Deploy affected apps run: | if pnpm nx affected:apps --base=origin/main --head=HEAD | grep -q "web"; then echo "Deploying web app" pnpm --filter web deploy fi ``` ## Best Practices 1. **Consistent Versioning**: Lock dependency versions across workspace 2. **Shared Configs**: Centralize ESLint, TypeScript, Prettier configs 3. **Dependency Graph**: Keep it acyclic, avoid circular dependencies 4. **Cache Effectively**: Configure inputs/outputs correctly 5. **Type Safety**: Share types between frontend/backend 6. **Testing Strategy**: Unit tests in packages, E2E in apps 7. **Documentation**: README in each package 8. **Release Strategy**: Use changesets for versioning ## Common Pitfalls - **Circular Dependencies**: A depends on B, B depends on A - **Phantom Dependencies**: Using deps not in package.json - **Incorrect Cache Inputs**: Missing files in Turborepo inputs - **Over-Sharing**: Sharing code that should be separate - **Under-Sharing**: Duplicating code across packages - **Large Monorepos**: Without proper tooling, builds slow down ## Publishing Packages ```bash # Using Changesets pnpm add -Dw @changesets/cli pnpm changeset init # Create changeset pnpm changeset # Version packages pnpm changeset version # Publish pnpm changeset publish ``` ```yaml # .github/workflows/release.yml - name: Create Release Pull Request or Publish uses: changesets/action@v1 with: publish: pnpm release env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} NPM_TOKEN: ${{ secrets.NPM_TOKEN }} ``` ## Resources - **references/turborepo-guide.md**: Comprehensive Turborepo documentation - **references/nx-guide.md**: Nx monorepo patterns - **references/pnpm-workspaces.md**: pnpm workspace features - **assets/monorepo-checklist.md**: Setup checklist - **assets/migration-guide.md**: Multi-repo to monorepo migration - **scripts/dependency-graph.ts**: Visualize package dependencies
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

nx-workspace-patterns

Configure and optimize Nx monorepo workspaces. Use when setting up

coding
⭐1
# Nx Workspace Patterns Production patterns for Nx monorepo management. ## When to Use This Skill - Setting up new Nx workspaces - Configuring project boundaries - Optimizing CI with affected commands - Implementing remote caching - Managing dependencies between projects - Migrating to Nx ## Core Concepts ### 1. Nx Architecture ``` workspace/ β”œβ”€β”€ apps/ # Deployable applications β”‚ β”œβ”€β”€ web/ β”‚ └── api/ β”œβ”€β”€ libs/ # Shared libraries β”‚ β”œβ”€β”€ shared/ β”‚ β”‚ β”œβ”€β”€ ui/ β”‚ β”‚ └── utils/ β”‚ └── feature/ β”‚ β”œβ”€β”€ auth/ β”‚ └── dashboard/ β”œβ”€β”€ tools/ # Custom executors/generators β”œβ”€β”€ nx.json # Nx configuration └── workspace.json # Project configuration ``` ### 2. Library Types | Type | Purpose | Example | | --------------- | -------------------------------- | ------------------- | | **feature** | Smart components, business logic | `feature-auth` | | **ui** | Presentational components | `ui-buttons` | | **data-access** | API calls, state management | `data-access-users` | | **util** | Pure functions, helpers | `util-formatting` | | **shell** | App bootstrapping | `shell-web` | ## Templates ### Template 1: nx.json Configuration ```json { "$schema": "./node_modules/nx/schemas/nx-schema.json", "npmScope": "myorg", "affected": { "defaultBase": "main" }, "tasksRunnerOptions": { "default": { "runner": "nx/tasks-runners/default", "options": { "cacheableOperations": [ "build", "lint", "test", "e2e", "build-storybook" ], "parallel": 3 } } }, "targetDefaults": { "build": { "dependsOn": ["^build"], "inputs": ["production", "^production"], "cache": true }, "test": { "inputs": ["default", "^production", "{workspaceRoot}/jest.preset.js"], "cache": true }, "lint": { "inputs": ["default", "{workspaceRoot}/.eslintrc.json"], "cache": true }, "e2e": { "inputs": ["default", "^production"], "cache": true } }, "namedInputs": { "default": ["{projectRoot}/**/*", "sharedGlobals"], "production": [ "default", "!{projectRoot}/**/?(*.)+(spec|test).[jt]s?(x)?(.snap)", "!{projectRoot}/tsconfig.spec.json", "!{projectRoot}/jest.config.[jt]s", "!{projectRoot}/.eslintrc.json" ], "sharedGlobals": [ "{workspaceRoot}/babel.config.json", "{workspaceRoot}/tsconfig.base.json" ] }, "generators": { "@nx/react": { "application": { "style": "css", "linter": "eslint", "bundler": "webpack" }, "library": { "style": "css", "linter": "eslint" }, "component": { "style": "css" } } } } ``` ### Template 2: Project Configuration ```json // apps/web/project.json { "name": "web", "$schema": "../../node_modules/nx/schemas/project-schema.json", "sourceRoot": "apps/web/src", "projectType": "application", "tags": ["type:app", "scope:web"], "targets": { "build": { "executor": "@nx/webpack:webpack", "outputs": ["{options.outputPath}"], "defaultConfiguration": "production", "options": { "compiler": "babel", "outputPath": "dist/apps/web", "index": "apps/web/src/index.html", "main": "apps/web/src/main.tsx", "tsConfig": "apps/web/tsconfig.app.json", "assets": ["apps/web/src/assets"], "styles": ["apps/web/src/styles.css"] }, "configurations": { "development": { "extractLicenses": false, "optimization": false, "sourceMap": true }, "production": { "optimization": true, "outputHashing": "all", "sourceMap": false, "extractLicenses": true } } }, "serve": { "executor": "@nx/webpack:dev-server", "defaultConfiguration": "development", "options": { "buildTarget": "web:build" }, "configurations": { "development": { "buildTarget": "web:build:development" }, "production": { "buildTarget": "web:build:production" } } }, "test": { "executor": "@nx/jest:jest", "outputs": ["{workspaceRoot}/coverage/{projectRoot}"], "options": { "jestConfig": "apps/web/jest.config.ts", "passWithNoTests": true } }, "lint": { "executor": "@nx/eslint:lint", "outputs": ["{options.outputFile}"], "options": { "lintFilePatterns": ["apps/web/**/*.{ts,tsx,js,jsx}"] } } } } ``` ### Template 3: Module Boundary Rules ```json // .eslintrc.json { "root": true, "ignorePatterns": ["**/*"], "plugins": ["@nx"], "overrides": [ { "files": ["*.ts", "*.tsx", "*.js", "*.jsx"], "rules": { "@nx/enforce-module-boundaries": [ "error", { "enforceBuildableLibDependency": true, "allow": [], "depConstraints": [ { "sourceTag": "type:app", "onlyDependOnLibsWithTags": [ "type:feature", "type:ui", "type:data-access", "type:util" ] }, { "sourceTag": "type:feature", "onlyDependOnLibsWithTags": [ "type:ui", "type:data-access", "type:util" ] }, { "sourceTag": "type:ui", "onlyDependOnLibsWithTags": ["type:ui", "type:util"] }, { "sourceTag": "type:data-access", "onlyDependOnLibsWithTags": ["type:data-access", "type:util"] }, { "sourceTag": "type:util", "onlyDependOnLibsWithTags": ["type:util"] }, { "sourceTag": "scope:web", "onlyDependOnLibsWithTags": ["scope:web", "scope:shared"] }, { "sourceTag": "scope:api", "onlyDependOnLibsWithTags": ["scope:api", "scope:shared"] }, { "sourceTag": "scope:shared", "onlyDependOnLibsWithTags": ["scope:shared"] } ] } ] } } ] } ``` ### Template 4: Custom Generator ```typescript // tools/generators/feature-lib/index.ts import { Tree, formatFiles, generateFiles, joinPathFragments, names, readProjectConfiguration, } from "@nx/devkit"; import { libraryGenerator } from "@nx/react"; interface FeatureLibraryGeneratorSchema { name: string; scope: string; directory?: string; } export default async function featureLibraryGenerator( tree: Tree, options: FeatureLibraryGeneratorSchema, ) { const { name, scope, directory } = options; const projectDirectory = directory ? `${directory}/${name}` : `libs/${scope}/feature-${name}`; // Generate base library await libraryGenerator(tree, { name: `feature-${name}`, directory: projectDirectory, tags: `type:feature,scope:${scope}`, style: "css", skipTsConfig: false, skipFormat: true, unitTestRunner: "jest", linter: "eslint", }); // Add custom files const projectConfig = readProjectConfiguration( tree, `${scope}-feature-${name}`, ); const projectNames = names(name); generateFiles( tree, joinPathFragments(__dirname, "files"), projectConfig.sourceRoot, { ...projectNames, scope, tmpl: "", }, ); await formatFiles(tree); } ``` ### Template 5: CI Configuration with Affected ```yaml # .github/workflows/ci.yml name: CI on: push: branches: [main] pull_request: branches: [main] env: NX_CLOUD_ACCESS_TOKEN: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }} jobs: main: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 with: fetch-depth: 0 - uses: actions/setup-node@v4 with: node-version: 20 cache: "npm" - name: Install dependencies run: npm ci - name: Derive SHAs for affected commands uses: nrwl/nx-set-shas@v4 - name: Run affected lint run: npx nx affected -t lint --parallel=3 - name: Run affected test run: npx nx affected -t test --parallel=3 --configuration=ci - name: Run affected build run: npx nx affected -t build --parallel=3 - name: Run affected e2e run: npx nx affected -t e2e --parallel=1 ``` ### Template 6: Remote Caching Setup ```typescript // nx.json with Nx Cloud { "tasksRunnerOptions": { "default": { "runner": "nx-cloud", "options": { "cacheableOperations": ["build", "lint", "test", "e2e"], "accessToken": "your-nx-cloud-token", "parallel": 3, "cacheDirectory": ".nx/cache" } } }, "nxCloudAccessToken": "your-nx-cloud-token" } // Self-hosted cache with S3 { "tasksRunnerOptions": { "default": { "runner": "@nx-aws-cache/nx-aws-cache", "options": { "cacheableOperations": ["build", "lint", "test"], "awsRegion": "us-east-1", "awsBucket": "my-nx-cache-bucket", "awsProfile": "default" } } } } ``` ## Common Commands ```bash # Generate new library nx g @nx/react:lib feature-auth --directory=libs/web --tags=type:feature,scope:web # Run affected tests nx affected -t test --base=main # View dependency graph nx graph # Run specific project nx build web --configuration=production # Reset cache nx reset # Run migrations nx migrate latest nx migrate --run-migrations ``` ## Best Practices ### Do's - **Use tags consistently** - Enforce with module boundaries - **Enable caching early** - Significant CI savings - **Keep libs focused** - Single responsibility - **Use generators** - Ensure consistency - **Document boundaries** - Help new developers ### Don'ts - **Don't create circular deps** - Graph should be acyclic - **Don't skip affected** - Test only what changed - **Don't ignore boundaries** - Tech debt accumulates - **Don't over-granularize** - Balance lib count ## Resources - [Nx Documentation](https://nx.dev/getting-started/intro) - [Module Boundaries](https://nx.dev/core-features/enforce-module-boundaries) - [Nx Cloud](https://nx.app/)
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

turborepo-caching

Configure Turborepo for efficient monorepo builds with local and

coding
⭐1
# Turborepo Caching Production patterns for Turborepo build optimization. ## When to Use This Skill - Setting up new Turborepo projects - Configuring build pipelines - Implementing remote caching - Optimizing CI/CD performance - Migrating from other monorepo tools - Debugging cache misses ## Core Concepts ### 1. Turborepo Architecture ``` Workspace Root/ β”œβ”€β”€ apps/ β”‚ β”œβ”€β”€ web/ β”‚ β”‚ └── package.json β”‚ └── docs/ β”‚ └── package.json β”œβ”€β”€ packages/ β”‚ β”œβ”€β”€ ui/ β”‚ β”‚ └── package.json β”‚ └── config/ β”‚ └── package.json β”œβ”€β”€ turbo.json └── package.json ``` ### 2. Pipeline Concepts | Concept | Description | | -------------- | -------------------------------- | | **dependsOn** | Tasks that must complete first | | **cache** | Whether to cache outputs | | **outputs** | Files to cache | | **inputs** | Files that affect cache key | | **persistent** | Long-running tasks (dev servers) | ## Templates ### Template 1: turbo.json Configuration ```json { "$schema": "https://turbo.build/schema.json", "globalDependencies": [".env", ".env.local"], "globalEnv": ["NODE_ENV", "VERCEL_URL"], "pipeline": { "build": { "dependsOn": ["^build"], "outputs": ["dist/**", ".next/**", "!.next/cache/**"], "env": ["API_URL", "NEXT_PUBLIC_*"] }, "test": { "dependsOn": ["build"], "outputs": ["coverage/**"], "inputs": ["src/**/*.tsx", "src/**/*.ts", "test/**/*.ts"] }, "lint": { "outputs": [], "cache": true }, "typecheck": { "dependsOn": ["^build"], "outputs": [] }, "dev": { "cache": false, "persistent": true }, "clean": { "cache": false } } } ``` ### Template 2: Package-Specific Pipeline ```json // apps/web/turbo.json { "$schema": "https://turbo.build/schema.json", "extends": ["//"], "pipeline": { "build": { "outputs": [".next/**", "!.next/cache/**"], "env": ["NEXT_PUBLIC_API_URL", "NEXT_PUBLIC_ANALYTICS_ID"] }, "test": { "outputs": ["coverage/**"], "inputs": ["src/**", "tests/**", "jest.config.js"] } } } ``` ### Template 3: Remote Caching with Vercel ```bash # Login to Vercel npx turbo login # Link to Vercel project npx turbo link # Run with remote cache turbo build --remote-only # CI environment variables TURBO_TOKEN=your-token TURBO_TEAM=your-team ``` ```yaml # .github/workflows/ci.yml name: CI on: push: branches: [main] pull_request: env: TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }} TURBO_TEAM: ${{ vars.TURBO_TEAM }} jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: 20 cache: "npm" - name: Install dependencies run: npm ci - name: Build run: npx turbo build --filter='...[origin/main]' - name: Test run: npx turbo test --filter='...[origin/main]' ``` ### Template 4: Self-Hosted Remote Cache ```typescript // Custom remote cache server (Express) import express from "express"; import { createReadStream, createWriteStream } from "fs"; import { mkdir } from "fs/promises"; import { join } from "path"; const app = express(); const CACHE_DIR = "./cache"; // Get artifact app.get("/v8/artifacts/:hash", async (req, res) => { const { hash } = req.params; const team = req.query.teamId || "default"; const filePath = join(CACHE_DIR, team, hash); try { const stream = createReadStream(filePath); stream.pipe(res); } catch { res.status(404).send("Not found"); } }); // Put artifact app.put("/v8/artifacts/:hash", async (req, res) => { const { hash } = req.params; const team = req.query.teamId || "default"; const dir = join(CACHE_DIR, team); const filePath = join(dir, hash); await mkdir(dir, { recursive: true }); const stream = createWriteStream(filePath); req.pipe(stream); stream.on("finish", () => { res.json({ urls: [`${req.protocol}://${req.get("host")}/v8/artifacts/${hash}`], }); }); }); // Check artifact exists app.head("/v8/artifacts/:hash", async (req, res) => { const { hash } = req.params; const team = req.query.teamId || "default"; const filePath = join(CACHE_DIR, team, hash); try { await fs.access(filePath); res.status(200).end(); } catch { res.status(404).end(); } }); app.listen(3000); ``` ```json // turbo.json for self-hosted cache { "remoteCache": { "signature": false } } ``` ```bash # Use self-hosted cache turbo build --api="http://localhost:3000" --token="my-token" --team="my-team" ``` ### Template 5: Filtering and Scoping ```bash # Build specific package turbo build --filter=@myorg/web # Build package and its dependencies turbo build --filter=@myorg/web... # Build package and its dependents turbo build --filter=...@myorg/ui # Build changed packages since main turbo build --filter='...[origin/main]' # Build packages in directory turbo build --filter='./apps/*' # Combine filters turbo build --filter=@myorg/web --filter=@myorg/docs # Exclude package turbo build --filter='!@myorg/docs' # Include dependencies of changed turbo build --filter='...[HEAD^1]...' ``` ### Template 6: Advanced Pipeline Configuration ```json { "$schema": "https://turbo.build/schema.json", "pipeline": { "build": { "dependsOn": ["^build"], "outputs": ["dist/**"], "inputs": ["$TURBO_DEFAULT$", "!**/*.md", "!**/*.test.*"] }, "test": { "dependsOn": ["^build"], "outputs": ["coverage/**"], "inputs": ["src/**", "tests/**", "*.config.*"], "env": ["CI", "NODE_ENV"] }, "test:e2e": { "dependsOn": ["build"], "outputs": [], "cache": false }, "deploy": { "dependsOn": ["build", "test", "lint"], "outputs": [], "cache": false }, "db:generate": { "cache": false }, "db:push": { "cache": false, "dependsOn": ["db:generate"] }, "@myorg/web#build": { "dependsOn": ["^build", "@myorg/db#db:generate"], "outputs": [".next/**"], "env": ["NEXT_PUBLIC_*"] } } } ``` ### Template 7: Root package.json Setup ```json { "name": "my-turborepo", "private": true, "workspaces": ["apps/*", "packages/*"], "scripts": { "build": "turbo build", "dev": "turbo dev", "lint": "turbo lint", "test": "turbo test", "clean": "turbo clean && rm -rf node_modules", "format": "prettier --write \"**/*.{ts,tsx,md}\"", "changeset": "changeset", "version-packages": "changeset version", "release": "turbo build --filter=./packages/* && changeset publish" }, "devDependencies": { "turbo": "^1.10.0", "prettier": "^3.0.0", "@changesets/cli": "^2.26.0" }, "packageManager": "npm@10.0.0" } ``` ## Debugging Cache ```bash # Dry run to see what would run turbo build --dry-run # Verbose output with hashes turbo build --verbosity=2 # Show task graph turbo build --graph # Force no cache turbo build --force # Show cache status turbo build --summarize # Debug specific task TURBO_LOG_VERBOSITY=debug turbo build --filter=@myorg/web ``` ## Best Practices ### Do's - **Define explicit inputs** - Avoid cache invalidation - **Use workspace protocol** - `"@myorg/ui": "workspace:*"` - **Enable remote caching** - Share across CI and local - **Filter in CI** - Build only affected packages - **Cache build outputs** - Not source files ### Don'ts - **Don't cache dev servers** - Use `persistent: true` - **Don't include secrets in env** - Use runtime env vars - **Don't ignore dependsOn** - Causes race conditions - **Don't over-filter** - May miss dependencies ## Resources - [Turborepo Documentation](https://turbo.build/repo/docs) - [Caching Guide](https://turbo.build/repo/docs/core-concepts/caching) - [Remote Caching](https://turbo.build/repo/docs/core-concepts/remote-caching)
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

dotnet-backend-patterns

Master C#/.NET backend development patterns for building robust

coding
⭐1
# .NET Backend Development Patterns Master C#/.NET patterns for building production-grade APIs, MCP servers, and enterprise backends with modern best practices (2024/2025). ## When to Use This Skill - Developing new .NET Web APIs or MCP servers - Reviewing C# code for quality and performance - Designing service architectures with dependency injection - Implementing caching strategies with Redis - Writing unit and integration tests - Optimizing database access with EF Core or Dapper - Configuring applications with IOptions pattern - Handling errors and implementing resilience patterns ## Core Concepts ### 1. Project Structure (Clean Architecture) ``` src/ β”œβ”€β”€ Domain/ # Core business logic (no dependencies) β”‚ β”œβ”€β”€ Entities/ β”‚ β”œβ”€β”€ Interfaces/ β”‚ β”œβ”€β”€ Exceptions/ β”‚ └── ValueObjects/ β”œβ”€β”€ Application/ # Use cases, DTOs, validation β”‚ β”œβ”€β”€ Services/ β”‚ β”œβ”€β”€ DTOs/ β”‚ β”œβ”€β”€ Validators/ β”‚ └── Interfaces/ β”œβ”€β”€ Infrastructure/ # External implementations β”‚ β”œβ”€β”€ Data/ # EF Core, Dapper repositories β”‚ β”œβ”€β”€ Caching/ # Redis, Memory cache β”‚ β”œβ”€β”€ External/ # HTTP clients, third-party APIs β”‚ └── DependencyInjection/ # Service registration └── Api/ # Entry point β”œβ”€β”€ Controllers/ # Or MinimalAPI endpoints β”œβ”€β”€ Middleware/ β”œβ”€β”€ Filters/ └── Program.cs ``` ### 2. Dependency Injection Patterns ```csharp // Service registration by lifetime public static class ServiceCollectionExtensions { public static IServiceCollection AddApplicationServices( this IServiceCollection services, IConfiguration configuration) { // Scoped: One instance per HTTP request services.AddScoped<IProductService, ProductService>(); services.AddScoped<IOrderService, OrderService>(); // Singleton: One instance for app lifetime services.AddSingleton<ICacheService, RedisCacheService>(); services.AddSingleton<IConnectionMultiplexer>(_ => ConnectionMultiplexer.Connect(configuration["Redis:Connection"]!)); // Transient: New instance every time services.AddTransient<IValidator<CreateOrderRequest>, CreateOrderValidator>(); // Options pattern for configuration services.Configure<CatalogOptions>(configuration.GetSection("Catalog")); services.Configure<RedisOptions>(configuration.GetSection("Redis")); // Factory pattern for conditional creation services.AddScoped<IPriceCalculator>(sp => { var options = sp.GetRequiredService<IOptions<PricingOptions>>().Value; return options.UseNewEngine ? sp.GetRequiredService<NewPriceCalculator>() : sp.GetRequiredService<LegacyPriceCalculator>(); }); // Keyed services (.NET 8+) services.AddKeyedScoped<IPaymentProcessor, StripeProcessor>("stripe"); services.AddKeyedScoped<IPaymentProcessor, PayPalProcessor>("paypal"); return services; } } // Usage with keyed services public class CheckoutService { public CheckoutService( [FromKeyedServices("stripe")] IPaymentProcessor stripeProcessor) { _processor = stripeProcessor; } } ``` ### 3. Async/Await Patterns ```csharp // βœ… CORRECT: Async all the way down public async Task<Product> GetProductAsync(string id, CancellationToken ct = default) { return await _repository.GetByIdAsync(id, ct); } // βœ… CORRECT: Parallel execution with WhenAll public async Task<(Stock, Price)> GetStockAndPriceAsync( string productId, CancellationToken ct = default) { var stockTask = _stockService.GetAsync(productId, ct); var priceTask = _priceService.GetAsync(productId, ct); await Task.WhenAll(stockTask, priceTask); return (await stockTask, await priceTask); } // βœ… CORRECT: ConfigureAwait in libraries public async Task<T> LibraryMethodAsync<T>(CancellationToken ct = default) { var result = await _httpClient.GetAsync(url, ct).ConfigureAwait(false); return await result.Content.ReadFromJsonAsync<T>(ct).ConfigureAwait(false); } // βœ… CORRECT: ValueTask for hot paths with caching public ValueTask<Product?> GetCachedProductAsync(string id) { if (_cache.TryGetValue(id, out Product? product)) return ValueTask.FromResult(product); return new ValueTask<Product?>(GetFromDatabaseAsync(id)); } // ❌ WRONG: Blocking on async (deadlock risk) var result = GetProductAsync(id).Result; // NEVER do this var result2 = GetProductAsync(id).GetAwaiter().GetResult(); // Also bad // ❌ WRONG: async void (except event handlers) public async void ProcessOrder() { } // Exceptions are lost // ❌ WRONG: Unnecessary Task.Run for already async code await Task.Run(async () => await GetDataAsync()); // Wastes thread ``` ### 4. Configuration with IOptions ```csharp // Configuration classes public class CatalogOptions { public const string SectionName = "Catalog"; public int DefaultPageSize { get; set; } = 50; public int MaxPageSize { get; set; } = 200; public TimeSpan CacheDuration { get; set; } = TimeSpan.FromMinutes(15); public bool EnableEnrichment { get; set; } = true; } public class RedisOptions { public const string SectionName = "Redis"; public string Connection { get; set; } = "localhost:6379"; public string KeyPrefix { get; set; } = "mcp:"; public int Database { get; set; } = 0; } // appsettings.json { "Catalog": { "DefaultPageSize": 50, "MaxPageSize": 200, "CacheDuration": "00:15:00", "EnableEnrichment": true }, "Redis": { "Connection": "localhost:6379", "KeyPrefix": "mcp:", "Database": 0 } } // Registration services.Configure<CatalogOptions>(configuration.GetSection(CatalogOptions.SectionName)); services.Configure<RedisOptions>(configuration.GetSection(RedisOptions.SectionName)); // Usage with IOptions (singleton, read once at startup) public class CatalogService { private readonly CatalogOptions _options; public CatalogService(IOptions<CatalogOptions> options) { _options = options.Value; } } // Usage with IOptionsSnapshot (scoped, re-reads on each request) public class DynamicService { private readonly CatalogOptions _options; public DynamicService(IOptionsSnapshot<CatalogOptions> options) { _options = options.Value; // Fresh value per request } } // Usage with IOptionsMonitor (singleton, notified on changes) public class MonitoredService { private CatalogOptions _options; public MonitoredService(IOptionsMonitor<CatalogOptions> monitor) { _options = monitor.CurrentValue; monitor.OnChange(newOptions => _options = newOptions); } } ``` ### 5. Result Pattern (Avoiding Exceptions for Flow Control) ```csharp // Generic Result type public class Result<T> { public bool IsSuccess { get; } public T? Value { get; } public string? Error { get; } public string? ErrorCode { get; } private Result(bool isSuccess, T? value, string? error, string? errorCode) { IsSuccess = isSuccess; Value = value; Error = error; ErrorCode = errorCode; } public static Result<T> Success(T value) => new(true, value, null, null); public static Result<T> Failure(string error, string? code = null) => new(false, default, error, code); public Result<TNew> Map<TNew>(Func<T, TNew> mapper) => IsSuccess ? Result<TNew>.Success(mapper(Value!)) : Result<TNew>.Failure(Error!, ErrorCode); public async Task<Result<TNew>> MapAsync<TNew>(Func<T, Task<TNew>> mapper) => IsSuccess ? Result<TNew>.Success(await mapper(Value!)) : Result<TNew>.Failure(Error!, ErrorCode); } // Usage in service public async Task<Result<Order>> CreateOrderAsync(CreateOrderRequest request, CancellationToken ct) { // Validation var validation = await _validator.ValidateAsync(request, ct); if (!validation.IsValid) return Result<Order>.Failure( validation.Errors.First().ErrorMessage, "VALIDATION_ERROR"); // Business rule check var stock = await _stockService.CheckAsync(request.ProductId, request.Quantity, ct); if (!stock.IsAvailable) return Result<Order>.Failure( $"Insufficient stock: {stock.Available} available, {request.Quantity} requested", "INSUFFICIENT_STOCK"); // Create order var order = await _repository.CreateAsync(request.ToEntity(), ct); return Result<Order>.Success(order); } // Usage in controller/endpoint app.MapPost("/orders", async ( CreateOrderRequest request, IOrderService orderService, CancellationToken ct) => { var result = await orderService.CreateOrderAsync(request, ct); return result.IsSuccess ? Results.Created($"/orders/{result.Value!.Id}", result.Value) : Results.BadRequest(new { error = result.Error, code = result.ErrorCode }); }); ``` ## Data Access Patterns ### Entity Framework Core ```csharp // DbContext configuration public class AppDbContext : DbContext { public DbSet<Product> Products => Set<Product>(); public DbSet<Order> Orders => Set<Order>(); protected override void OnModelCreating(ModelBuilder modelBuilder) { // Apply all configurations from assembly modelBuilder.ApplyConfigurationsFromAssembly(typeof(AppDbContext).Assembly); // Global query filters modelBuilder.Entity<Product>().HasQueryFilter(p => !p.IsDeleted); } } // Entity configuration public class ProductConfiguration : IEntityTypeConfiguration<Product> { public void Configure(EntityTypeBuilder<Product> builder) { builder.ToTable("Products"); builder.HasKey(p => p.Id); builder.Property(p => p.Id).HasMaxLength(40); builder.Property(p => p.Name).HasMaxLength(200).IsRequired(); builder.Property(p => p.Price).HasPrecision(18, 2); builder.HasIndex(p => p.Sku).IsUnique(); builder.HasIndex(p => new { p.CategoryId, p.Name }); builder.HasMany(p => p.OrderItems) .WithOne(oi => oi.Product) .HasForeignKey(oi => oi.ProductId); } } // Repository with EF Core public class ProductRepository : IProductRepository { private readonly AppDbContext _context; public async Task<Product?> GetByIdAsync(string id, CancellationToken ct = default) { return await _context.Products .AsNoTracking() .FirstOrDefaultAsync(p => p.Id == id, ct); } public async Task<IReadOnlyList<Product>> SearchAsync( ProductSearchCriteria criteria, CancellationToken ct = default) { var query = _context.Products.AsNoTracking(); if (!string.IsNullOrWhiteSpace(criteria.SearchTerm)) query = query.Where(p => EF.Functions.Like(p.Name, $"%{criteria.SearchTerm}%")); if (criteria.CategoryId.HasValue) query = query.Where(p => p.CategoryId == criteria.CategoryId); if (criteria.MinPrice.HasValue) query = query.Where(p => p.Price >= criteria.MinPrice); if (criteria.MaxPrice.HasValue) query = query.Where(p => p.Price <= criteria.MaxPrice); return await query .OrderBy(p => p.Name) .Skip((criteria.Page - 1) * criteria.PageSize) .Take(criteria.PageSize) .ToListAsync(ct); } } ``` ### Dapper for Performance ```csharp public class DapperProductRepository : IProductRepository { private readonly IDbConnection _connection; public async Task<Product?> GetByIdAsync(string id, CancellationToken ct = default) { const string sql = """ SELECT Id, Name, Sku, Price, CategoryId, Stock, CreatedAt FROM Products WHERE Id = @Id AND IsDeleted = 0 """; return await _connection.QueryFirstOrDefaultAsync<Product>( new CommandDefinition(sql, new { Id = id }, cancellationToken: ct)); } public async Task<IReadOnlyList<Product>> SearchAsync( ProductSearchCriteria criteria, CancellationToken ct = default) { var sql = new StringBuilder(""" SELECT Id, Name, Sku, Price, CategoryId, Stock, CreatedAt FROM Products WHERE IsDeleted = 0 """); var parameters = new DynamicParameters(); if (!string.IsNullOrWhiteSpace(criteria.SearchTerm)) { sql.Append(" AND Name LIKE @SearchTerm"); parameters.Add("SearchTerm", $"%{criteria.SearchTerm}%"); } if (criteria.CategoryId.HasValue) { sql.Append(" AND CategoryId = @CategoryId"); parameters.Add("CategoryId", criteria.CategoryId); } if (criteria.MinPrice.HasValue) { sql.Append(" AND Price >= @MinPrice"); parameters.Add("MinPrice", criteria.MinPrice); } if (criteria.MaxPrice.HasValue) { sql.Append(" AND Price <= @MaxPrice"); parameters.Add("MaxPrice", criteria.MaxPrice); } sql.Append(" ORDER BY Name OFFSET @Offset ROWS FETCH NEXT @PageSize ROWS ONLY"); parameters.Add("Offset", (criteria.Page - 1) * criteria.PageSize); parameters.Add("PageSize", criteria.PageSize); var results = await _connection.QueryAsync<Product>( new CommandDefinition(sql.ToString(), parameters, cancellationToken: ct)); return results.ToList(); } // Multi-mapping for related data public async Task<Order?> GetOrderWithItemsAsync(int orderId, CancellationToken ct = default) { const string sql = """ SELECT o.*, oi.*, p.* FROM Orders o LEFT JOIN OrderItems oi ON o.Id = oi.OrderId LEFT JOIN Products p ON oi.ProductId = p.Id WHERE o.Id = @OrderId """; var orderDictionary = new Dictionary<int, Order>(); await _connection.QueryAsync<Order, OrderItem, Product, Order>( new CommandDefinition(sql, new { OrderId = orderId }, cancellationToken: ct), (order, item, product) => { if (!orderDictionary.TryGetValue(order.Id, out var existingOrder)) { existingOrder = order; existingOrder.Items = new List<OrderItem>(); orderDictionary.Add(order.Id, existingOrder); } if (item != null) { item.Product = product; existingOrder.Items.Add(item); } return existingOrder; }, splitOn: "Id,Id"); return orderDictionary.Values.FirstOrDefault(); } } ``` ## Caching Patterns ### Multi-Level Cache with Redis ```csharp public class CachedProductService : IProductService { private readonly IProductRepository _repository; private readonly IMemoryCache _memoryCache; private readonly IDistributedCache _distributedCache; private readonly ILogger<CachedProductService> _logger; private static readonly TimeSpan MemoryCacheDuration = TimeSpan.FromMinutes(1); private static readonly TimeSpan DistributedCacheDuration = TimeSpan.FromMinutes(15); public async Task<Product?> GetByIdAsync(string id, CancellationToken ct = default) { var cacheKey = $"product:{id}"; // L1: Memory cache (in-process, fastest) if (_memoryCache.TryGetValue(cacheKey, out Product? cached)) { _logger.LogDebug("L1 cache hit for {CacheKey}", cacheKey); return cached; } // L2: Distributed cache (Redis) var distributed = await _distributedCache.GetStringAsync(cacheKey, ct); if (distributed != null) { _logger.LogDebug("L2 cache hit for {CacheKey}", cacheKey); var product = JsonSerializer.Deserialize<Product>(distributed); // Populate L1 _memoryCache.Set(cacheKey, product, MemoryCacheDuration); return product; } // L3: Database _logger.LogDebug("Cache miss for {CacheKey}, fetching from database", cacheKey); var fromDb = await _repository.GetByIdAsync(id, ct); if (fromDb != null) { var serialized = JsonSerializer.Serialize(fromDb); // Populate both caches await _distributedCache.SetStringAsync( cacheKey, serialized, new DistributedCacheEntryOptions { AbsoluteExpirationRelativeToNow = DistributedCacheDuration }, ct); _memoryCache.Set(cacheKey, fromDb, MemoryCacheDuration); } return fromDb; } public async Task InvalidateAsync(string id, CancellationToken ct = default) { var cacheKey = $"product:{id}"; _memoryCache.Remove(cacheKey); await _distributedCache.RemoveAsync(cacheKey, ct); _logger.LogInformation("Invalidated cache for {CacheKey}", cacheKey); } } // Stale-while-revalidate pattern public class StaleWhileRevalidateCache<T> { private readonly IDistributedCache _cache; private readonly TimeSpan _freshDuration; private readonly TimeSpan _staleDuration; public async Task<T?> GetOrCreateAsync( string key, Func<CancellationToken, Task<T>> factory, CancellationToken ct = default) { var cached = await _cache.GetStringAsync(key, ct); if (cached != null) { var entry = JsonSerializer.Deserialize<CacheEntry<T>>(cached)!; if (entry.IsStale && !entry.IsExpired) { // Return stale data immediately, refresh in background _ = Task.Run(async () => { var fresh = await factory(CancellationToken.None); await SetAsync(key, fresh, CancellationToken.None); }); } if (!entry.IsExpired) return entry.Value; } // Cache miss or expired var value = await factory(ct); await SetAsync(key, value, ct); return value; } private record CacheEntry<TValue>(TValue Value, DateTime CreatedAt) { public bool IsStale => DateTime.UtcNow - CreatedAt > _freshDuration; public bool IsExpired => DateTime.UtcNow - CreatedAt > _staleDuration; } } ``` ## Testing Patterns ### Unit Tests with xUnit and Moq ```csharp public class OrderServiceTests { private readonly Mock<IOrderRepository> _mockRepository; private readonly Mock<IStockService> _mockStockService; private readonly Mock<IValidator<CreateOrderRequest>> _mockValidator; private readonly OrderService _sut; // System Under Test public OrderServiceTests() { _mockRepository = new Mock<IOrderRepository>(); _mockStockService = new Mock<IStockService>(); _mockValidator = new Mock<IValidator<CreateOrderRequest>>(); // Default: validation passes _mockValidator .Setup(v => v.ValidateAsync(It.IsAny<CreateOrderRequest>(), It.IsAny<
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

react-native-architecture

Build production React Native apps with Expo, navigation, native

coding
⭐1
# React Native Architecture Production-ready patterns for React Native development with Expo, including navigation, state management, native modules, and offline-first architecture. ## When to Use This Skill - Starting a new React Native or Expo project - Implementing complex navigation patterns - Integrating native modules and platform APIs - Building offline-first mobile applications - Optimizing React Native performance - Setting up CI/CD for mobile releases ## Core Concepts ### 1. Project Structure ``` src/ β”œβ”€β”€ app/ # Expo Router screens β”‚ β”œβ”€β”€ (auth)/ # Auth group β”‚ β”œβ”€β”€ (tabs)/ # Tab navigation β”‚ └── _layout.tsx # Root layout β”œβ”€β”€ components/ β”‚ β”œβ”€β”€ ui/ # Reusable UI components β”‚ └── features/ # Feature-specific components β”œβ”€β”€ hooks/ # Custom hooks β”œβ”€β”€ services/ # API and native services β”œβ”€β”€ stores/ # State management β”œβ”€β”€ utils/ # Utilities └── types/ # TypeScript types ``` ### 2. Expo vs Bare React Native | Feature | Expo | Bare RN | | ------------------ | -------------- | -------------- | | Setup complexity | Low | High | | Native modules | EAS Build | Manual linking | | OTA updates | Built-in | Manual setup | | Build service | EAS | Custom CI | | Custom native code | Config plugins | Direct access | ## Quick Start ```bash # Create new Expo project npx create-expo-app@latest my-app -t expo-template-blank-typescript # Install essential dependencies npx expo install expo-router expo-status-bar react-native-safe-area-context npx expo install @react-native-async-storage/async-storage npx expo install expo-secure-store expo-haptics ``` ```typescript // app/_layout.tsx import { Stack } from 'expo-router' import { ThemeProvider } from '@/providers/ThemeProvider' import { QueryProvider } from '@/providers/QueryProvider' export default function RootLayout() { return ( <QueryProvider> <ThemeProvider> <Stack screenOptions={{ headerShown: false }}> <Stack.Screen name="(tabs)" /> <Stack.Screen name="(auth)" /> <Stack.Screen name="modal" options={{ presentation: 'modal' }} /> </Stack> </ThemeProvider> </QueryProvider> ) } ``` ## Patterns ### Pattern 1: Expo Router Navigation ```typescript // app/(tabs)/_layout.tsx import { Tabs } from 'expo-router' import { Home, Search, User, Settings } from 'lucide-react-native' import { useTheme } from '@/hooks/useTheme' export default function TabLayout() { const { colors } = useTheme() return ( <Tabs screenOptions={{ tabBarActiveTintColor: colors.primary, tabBarInactiveTintColor: colors.textMuted, tabBarStyle: { backgroundColor: colors.background }, headerShown: false, }} > <Tabs.Screen name="index" options={{ title: 'Home', tabBarIcon: ({ color, size }) => <Home size={size} color={color} />, }} /> <Tabs.Screen name="search" options={{ title: 'Search', tabBarIcon: ({ color, size }) => <Search size={size} color={color} />, }} /> <Tabs.Screen name="profile" options={{ title: 'Profile', tabBarIcon: ({ color, size }) => <User size={size} color={color} />, }} /> <Tabs.Screen name="settings" options={{ title: 'Settings', tabBarIcon: ({ color, size }) => <Settings size={size} color={color} />, }} /> </Tabs> ) } // app/(tabs)/profile/[id].tsx - Dynamic route import { useLocalSearchParams } from 'expo-router' export default function ProfileScreen() { const { id } = useLocalSearchParams<{ id: string }>() return <UserProfile userId={id} /> } // Navigation from anywhere import { router } from 'expo-router' // Programmatic navigation router.push('/profile/123') router.replace('/login') router.back() // With params router.push({ pathname: '/product/[id]', params: { id: '123', referrer: 'home' }, }) ``` ### Pattern 2: Authentication Flow ```typescript // providers/AuthProvider.tsx import { createContext, useContext, useEffect, useState } from 'react' import { useRouter, useSegments } from 'expo-router' import * as SecureStore from 'expo-secure-store' interface AuthContextType { user: User | null isLoading: boolean signIn: (credentials: Credentials) => Promise<void> signOut: () => Promise<void> } const AuthContext = createContext<AuthContextType | null>(null) export function AuthProvider({ children }: { children: React.ReactNode }) { const [user, setUser] = useState<User | null>(null) const [isLoading, setIsLoading] = useState(true) const segments = useSegments() const router = useRouter() // Check authentication on mount useEffect(() => { checkAuth() }, []) // Protect routes useEffect(() => { if (isLoading) return const inAuthGroup = segments[0] === '(auth)' if (!user && !inAuthGroup) { router.replace('/login') } else if (user && inAuthGroup) { router.replace('/(tabs)') } }, [user, segments, isLoading]) async function checkAuth() { try { const token = await SecureStore.getItemAsync('authToken') if (token) { const userData = await api.getUser(token) setUser(userData) } } catch (error) { await SecureStore.deleteItemAsync('authToken') } finally { setIsLoading(false) } } async function signIn(credentials: Credentials) { const { token, user } = await api.login(credentials) await SecureStore.setItemAsync('authToken', token) setUser(user) } async function signOut() { await SecureStore.deleteItemAsync('authToken') setUser(null) } if (isLoading) { return <SplashScreen /> } return ( <AuthContext.Provider value={{ user, isLoading, signIn, signOut }}> {children} </AuthContext.Provider> ) } export const useAuth = () => { const context = useContext(AuthContext) if (!context) throw new Error('useAuth must be used within AuthProvider') return context } ``` ### Pattern 3: Offline-First with React Query ```typescript // providers/QueryProvider.tsx import { QueryClient, QueryClientProvider } from '@tanstack/react-query' import { createAsyncStoragePersister } from '@tanstack/query-async-storage-persister' import { PersistQueryClientProvider } from '@tanstack/react-query-persist-client' import AsyncStorage from '@react-native-async-storage/async-storage' import NetInfo from '@react-native-community/netinfo' import { onlineManager } from '@tanstack/react-query' // Sync online status onlineManager.setEventListener((setOnline) => { return NetInfo.addEventListener((state) => { setOnline(!!state.isConnected) }) }) const queryClient = new QueryClient({ defaultOptions: { queries: { gcTime: 1000 * 60 * 60 * 24, // 24 hours staleTime: 1000 * 60 * 5, // 5 minutes retry: 2, networkMode: 'offlineFirst', }, mutations: { networkMode: 'offlineFirst', }, }, }) const asyncStoragePersister = createAsyncStoragePersister({ storage: AsyncStorage, key: 'REACT_QUERY_OFFLINE_CACHE', }) export function QueryProvider({ children }: { children: React.ReactNode }) { return ( <PersistQueryClientProvider client={queryClient} persistOptions={{ persister: asyncStoragePersister }} > {children} </PersistQueryClientProvider> ) } // hooks/useProducts.ts import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query' export function useProducts() { return useQuery({ queryKey: ['products'], queryFn: api.getProducts, // Use stale data while revalidating placeholderData: (previousData) => previousData, }) } export function useCreateProduct() { const queryClient = useQueryClient() return useMutation({ mutationFn: api.createProduct, // Optimistic update onMutate: async (newProduct) => { await queryClient.cancelQueries({ queryKey: ['products'] }) const previous = queryClient.getQueryData(['products']) queryClient.setQueryData(['products'], (old: Product[]) => [ ...old, { ...newProduct, id: 'temp-' + Date.now() }, ]) return { previous } }, onError: (err, newProduct, context) => { queryClient.setQueryData(['products'], context?.previous) }, onSettled: () => { queryClient.invalidateQueries({ queryKey: ['products'] }) }, }) } ``` ### Pattern 4: Native Module Integration ```typescript // services/haptics.ts import * as Haptics from "expo-haptics"; import { Platform } from "react-native"; export const haptics = { light: () => { if (Platform.OS !== "web") { Haptics.impactAsync(Haptics.ImpactFeedbackStyle.Light); } }, medium: () => { if (Platform.OS !== "web") { Haptics.impactAsync(Haptics.ImpactFeedbackStyle.Medium); } }, heavy: () => { if (Platform.OS !== "web") { Haptics.impactAsync(Haptics.ImpactFeedbackStyle.Heavy); } }, success: () => { if (Platform.OS !== "web") { Haptics.notificationAsync(Haptics.NotificationFeedbackType.Success); } }, error: () => { if (Platform.OS !== "web") { Haptics.notificationAsync(Haptics.NotificationFeedbackType.Error); } }, }; // services/biometrics.ts import * as LocalAuthentication from "expo-local-authentication"; export async function authenticateWithBiometrics(): Promise<boolean> { const hasHardware = await LocalAuthentication.hasHardwareAsync(); if (!hasHardware) return false; const isEnrolled = await LocalAuthentication.isEnrolledAsync(); if (!isEnrolled) return false; const result = await LocalAuthentication.authenticateAsync({ promptMessage: "Authenticate to continue", fallbackLabel: "Use passcode", disableDeviceFallback: false, }); return result.success; } // services/notifications.ts import * as Notifications from "expo-notifications"; import { Platform } from "react-native"; import Constants from "expo-constants"; Notifications.setNotificationHandler({ handleNotification: async () => ({ shouldShowAlert: true, shouldPlaySound: true, shouldSetBadge: true, }), }); export async function registerForPushNotifications() { let token: string | undefined; if (Platform.OS === "android") { await Notifications.setNotificationChannelAsync("default", { name: "default", importance: Notifications.AndroidImportance.MAX, vibrationPattern: [0, 250, 250, 250], }); } const { status: existingStatus } = await Notifications.getPermissionsAsync(); let finalStatus = existingStatus; if (existingStatus !== "granted") { const { status } = await Notifications.requestPermissionsAsync(); finalStatus = status; } if (finalStatus !== "granted") { return null; } const projectId = Constants.expoConfig?.extra?.eas?.projectId; token = (await Notifications.getExpoPushTokenAsync({ projectId })).data; return token; } ``` ### Pattern 5: Platform-Specific Code ```typescript // components/ui/Button.tsx import { Platform, Pressable, StyleSheet, Text, ViewStyle } from 'react-native' import * as Haptics from 'expo-haptics' import Animated, { useAnimatedStyle, useSharedValue, withSpring, } from 'react-native-reanimated' const AnimatedPressable = Animated.createAnimatedComponent(Pressable) interface ButtonProps { title: string onPress: () => void variant?: 'primary' | 'secondary' | 'outline' disabled?: boolean } export function Button({ title, onPress, variant = 'primary', disabled = false, }: ButtonProps) { const scale = useSharedValue(1) const animatedStyle = useAnimatedStyle(() => ({ transform: [{ scale: scale.value }], })) const handlePressIn = () => { scale.value = withSpring(0.95) if (Platform.OS !== 'web') { Haptics.impactAsync(Haptics.ImpactFeedbackStyle.Light) } } const handlePressOut = () => { scale.value = withSpring(1) } return ( <AnimatedPressable onPress={onPress} onPressIn={handlePressIn} onPressOut={handlePressOut} disabled={disabled} style={[ styles.button, styles[variant], disabled && styles.disabled, animatedStyle, ]} > <Text style={[styles.text, styles[`${variant}Text`]]}>{title}</Text> </AnimatedPressable> ) } // Platform-specific files // Button.ios.tsx - iOS-specific implementation // Button.android.tsx - Android-specific implementation // Button.web.tsx - Web-specific implementation // Or use Platform.select const styles = StyleSheet.create({ button: { paddingVertical: 12, paddingHorizontal: 24, borderRadius: 8, alignItems: 'center', ...Platform.select({ ios: { shadowColor: '#000', shadowOffset: { width: 0, height: 2 }, shadowOpacity: 0.1, shadowRadius: 4, }, android: { elevation: 4, }, }), }, primary: { backgroundColor: '#007AFF', }, secondary: { backgroundColor: '#5856D6', }, outline: { backgroundColor: 'transparent', borderWidth: 1, borderColor: '#007AFF', }, disabled: { opacity: 0.5, }, text: { fontSize: 16, fontWeight: '600', }, primaryText: { color: '#FFFFFF', }, secondaryText: { color: '#FFFFFF', }, outlineText: { color: '#007AFF', }, }) ``` ### Pattern 6: Performance Optimization ```typescript // components/ProductList.tsx import { FlashList } from '@shopify/flash-list' import { memo, useCallback } from 'react' interface ProductListProps { products: Product[] onProductPress: (id: string) => void } // Memoize list item const ProductItem = memo(function ProductItem({ item, onPress, }: { item: Product onPress: (id: string) => void }) { const handlePress = useCallback(() => onPress(item.id), [item.id, onPress]) return ( <Pressable onPress={handlePress} style={styles.item}> <FastImage source={{ uri: item.image }} style={styles.image} resizeMode="cover" /> <Text style={styles.title}>{item.name}</Text> <Text style={styles.price}>${item.price}</Text> </Pressable> ) }) export function ProductList({ products, onProductPress }: ProductListProps) { const renderItem = useCallback( ({ item }: { item: Product }) => ( <ProductItem item={item} onPress={onProductPress} /> ), [onProductPress] ) const keyExtractor = useCallback((item: Product) => item.id, []) return ( <FlashList data={products} renderItem={renderItem} keyExtractor={keyExtractor} estimatedItemSize={100} // Performance optimizations removeClippedSubviews={true} maxToRenderPerBatch={10} windowSize={5} // Pull to refresh onRefresh={onRefresh} refreshing={isRefreshing} /> ) } ``` ## EAS Build & Submit ```json // eas.json { "cli": { "version": ">= 5.0.0" }, "build": { "development": { "developmentClient": true, "distribution": "internal", "ios": { "simulator": true } }, "preview": { "distribution": "internal", "android": { "buildType": "apk" } }, "production": { "autoIncrement": true } }, "submit": { "production": { "ios": { "appleId": "your@email.com", "ascAppId": "123456789" }, "android": { "serviceAccountKeyPath": "./google-services.json" } } } } ``` ```bash # Build commands eas build --platform ios --profile development eas build --platform android --profile preview eas build --platform all --profile production # Submit to stores eas submit --platform ios eas submit --platform android # OTA updates eas update --branch production --message "Bug fixes" ``` ## Best Practices ### Do's - **Use Expo** - Faster development, OTA updates, managed native code - **FlashList over FlatList** - Better performance for long lists - **Memoize components** - Prevent unnecessary re-renders - **Use Reanimated** - 60fps animations on native thread - **Test on real devices** - Simulators miss real-world issues ### Don'ts - **Don't inline styles** - Use StyleSheet.create for performance - **Don't fetch in render** - Use useEffect or React Query - **Don't ignore platform differences** - Test on both iOS and Android - **Don't store secrets in code** - Use environment variables - **Don't skip error boundaries** - Mobile crashes are unforgiving ## Resources - [Expo Documentation](https://docs.expo.dev/) - [Expo Router](https://docs.expo.dev/router/introduction/) - [React Native Performance](https://reactnative.dev/docs/performance) - [FlashList](https://shopify.github.io/flash-list/)
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

gdpr-data-handling

Implement GDPR-compliant data handling with consent management,

data
⭐1
# GDPR Data Handling Practical implementation guide for GDPR-compliant data processing, consent management, and privacy controls. ## When to Use This Skill - Building systems that process EU personal data - Implementing consent management - Handling data subject requests (DSRs) - Conducting GDPR compliance reviews - Designing privacy-first architectures - Creating data processing agreements ## Core Concepts ### 1. Personal Data Categories | Category | Examples | Protection Level | | ---------------------- | --------------------------- | ------------------ | | **Basic** | Name, email, phone | Standard | | **Sensitive (Art. 9)** | Health, religion, ethnicity | Explicit consent | | **Criminal (Art. 10)** | Convictions, offenses | Official authority | | **Children's** | Under 16 data | Parental consent | ### 2. Legal Bases for Processing ``` Article 6 - Lawful Bases: β”œβ”€β”€ Consent: Freely given, specific, informed β”œβ”€β”€ Contract: Necessary for contract performance β”œβ”€β”€ Legal Obligation: Required by law β”œβ”€β”€ Vital Interests: Protecting someone's life β”œβ”€β”€ Public Interest: Official functions └── Legitimate Interest: Balanced against rights ``` ### 3. Data Subject Rights ``` Right to Access (Art. 15) ─┐ Right to Rectification (Art. 16) β”‚ Right to Erasure (Art. 17) β”‚ Must respond Right to Restrict (Art. 18) β”‚ within 1 month Right to Portability (Art. 20) β”‚ Right to Object (Art. 21) β”€β”˜ ``` ## Implementation Patterns ### Pattern 1: Consent Management ```javascript // Consent data model const consentSchema = { userId: String, consents: [ { purpose: String, // 'marketing', 'analytics', etc. granted: Boolean, timestamp: Date, source: String, // 'web_form', 'api', etc. version: String, // Privacy policy version ipAddress: String, // For proof userAgent: String, // For proof }, ], auditLog: [ { action: String, // 'granted', 'withdrawn', 'updated' purpose: String, timestamp: Date, source: String, }, ], }; // Consent service class ConsentManager { async recordConsent(userId, purpose, granted, metadata) { const consent = { purpose, granted, timestamp: new Date(), source: metadata.source, version: await this.getCurrentPolicyVersion(), ipAddress: metadata.ipAddress, userAgent: metadata.userAgent, }; // Store consent await this.db.consents.updateOne( { userId }, { $push: { consents: consent, auditLog: { action: granted ? "granted" : "withdrawn", purpose, timestamp: consent.timestamp, source: metadata.source, }, }, }, { upsert: true }, ); // Emit event for downstream systems await this.eventBus.emit("consent.changed", { userId, purpose, granted, timestamp: consent.timestamp, }); } async hasConsent(userId, purpose) { const record = await this.db.consents.findOne({ userId }); if (!record) return false; const latestConsent = record.consents .filter((c) => c.purpose === purpose) .sort((a, b) => b.timestamp - a.timestamp)[0]; return latestConsent?.granted === true; } async getConsentHistory(userId) { const record = await this.db.consents.findOne({ userId }); return record?.auditLog || []; } } ``` ```html <!-- GDPR-compliant consent UI --> <div class="consent-banner" role="dialog" aria-labelledby="consent-title"> <h2 id="consent-title">Cookie Preferences</h2> <p> We use cookies to improve your experience. Select your preferences below. </p> <form id="consent-form"> <!-- Necessary - always on, no consent needed --> <div class="consent-category"> <input type="checkbox" id="necessary" checked disabled /> <label for="necessary"> <strong>Necessary</strong> <span>Required for the website to function. Cannot be disabled.</span> </label> </div> <!-- Analytics - requires consent --> <div class="consent-category"> <input type="checkbox" id="analytics" name="analytics" /> <label for="analytics"> <strong>Analytics</strong> <span>Help us understand how you use our site.</span> </label> </div> <!-- Marketing - requires consent --> <div class="consent-category"> <input type="checkbox" id="marketing" name="marketing" /> <label for="marketing"> <strong>Marketing</strong> <span>Personalized ads based on your interests.</span> </label> </div> <div class="consent-actions"> <button type="button" id="accept-all">Accept All</button> <button type="button" id="reject-all">Reject All</button> <button type="submit">Save Preferences</button> </div> <p class="consent-links"> <a href="/privacy-policy">Privacy Policy</a> | <a href="/cookie-policy">Cookie Policy</a> </p> </form> </div> ``` ### Pattern 2: Data Subject Access Request (DSAR) ```python from datetime import datetime, timedelta from typing import Dict, List, Optional import json class DSARHandler: """Handle Data Subject Access Requests.""" RESPONSE_DEADLINE_DAYS = 30 EXTENSION_ALLOWED_DAYS = 60 # For complex requests def __init__(self, data_sources: List['DataSource']): self.data_sources = data_sources async def submit_request( self, request_type: str, # 'access', 'erasure', 'rectification', 'portability' user_id: str, verified: bool, details: Optional[Dict] = None ) -> str: """Submit a new DSAR.""" request = { 'id': self.generate_request_id(), 'type': request_type, 'user_id': user_id, 'status': 'pending_verification' if not verified else 'processing', 'submitted_at': datetime.utcnow(), 'deadline': datetime.utcnow() + timedelta(days=self.RESPONSE_DEADLINE_DAYS), 'details': details or {}, 'audit_log': [{ 'action': 'submitted', 'timestamp': datetime.utcnow(), 'details': 'Request received' }] } await self.db.dsar_requests.insert_one(request) await self.notify_dpo(request) return request['id'] async def process_access_request(self, request_id: str) -> Dict: """Process a data access request.""" request = await self.get_request(request_id) if request['type'] != 'access': raise ValueError("Not an access request") # Collect data from all sources user_data = {} for source in self.data_sources: try: data = await source.get_user_data(request['user_id']) user_data[source.name] = data except Exception as e: user_data[source.name] = {'error': str(e)} # Format response response = { 'request_id': request_id, 'generated_at': datetime.utcnow().isoformat(), 'data_categories': list(user_data.keys()), 'data': user_data, 'retention_info': await self.get_retention_info(), 'processing_purposes': await self.get_processing_purposes(), 'third_party_recipients': await self.get_recipients() } # Update request status await self.update_request(request_id, 'completed', response) return response async def process_erasure_request(self, request_id: str) -> Dict: """Process a right to erasure request.""" request = await self.get_request(request_id) if request['type'] != 'erasure': raise ValueError("Not an erasure request") results = {} exceptions = [] for source in self.data_sources: try: # Check for legal exceptions can_delete, reason = await source.can_delete(request['user_id']) if can_delete: await source.delete_user_data(request['user_id']) results[source.name] = 'deleted' else: exceptions.append({ 'source': source.name, 'reason': reason # e.g., 'legal retention requirement' }) results[source.name] = f'retained: {reason}' except Exception as e: results[source.name] = f'error: {str(e)}' response = { 'request_id': request_id, 'completed_at': datetime.utcnow().isoformat(), 'results': results, 'exceptions': exceptions } await self.update_request(request_id, 'completed', response) return response async def process_portability_request(self, request_id: str) -> bytes: """Generate portable data export.""" request = await self.get_request(request_id) user_data = await self.process_access_request(request_id) # Convert to machine-readable format (JSON) portable_data = { 'export_date': datetime.utcnow().isoformat(), 'format_version': '1.0', 'data': user_data['data'] } return json.dumps(portable_data, indent=2, default=str).encode() ``` ### Pattern 3: Data Retention ```python from datetime import datetime, timedelta from enum import Enum class RetentionBasis(Enum): CONSENT = "consent" CONTRACT = "contract" LEGAL_OBLIGATION = "legal_obligation" LEGITIMATE_INTEREST = "legitimate_interest" class DataRetentionPolicy: """Define and enforce data retention policies.""" POLICIES = { 'user_account': { 'retention_period_days': 365 * 3, # 3 years after last activity 'basis': RetentionBasis.CONTRACT, 'trigger': 'last_activity_date', 'archive_before_delete': True }, 'transaction_records': { 'retention_period_days': 365 * 7, # 7 years for tax 'basis': RetentionBasis.LEGAL_OBLIGATION, 'trigger': 'transaction_date', 'archive_before_delete': True, 'legal_reference': 'Tax regulations require 7 year retention' }, 'marketing_consent': { 'retention_period_days': 365 * 2, # 2 years 'basis': RetentionBasis.CONSENT, 'trigger': 'consent_date', 'archive_before_delete': False }, 'support_tickets': { 'retention_period_days': 365 * 2, 'basis': RetentionBasis.LEGITIMATE_INTEREST, 'trigger': 'ticket_closed_date', 'archive_before_delete': True }, 'analytics_data': { 'retention_period_days': 365, # 1 year 'basis': RetentionBasis.CONSENT, 'trigger': 'collection_date', 'archive_before_delete': False, 'anonymize_instead': True } } async def apply_retention_policies(self): """Run retention policy enforcement.""" for data_type, policy in self.POLICIES.items(): cutoff_date = datetime.utcnow() - timedelta( days=policy['retention_period_days'] ) if policy.get('anonymize_instead'): await self.anonymize_old_data(data_type, cutoff_date) else: if policy.get('archive_before_delete'): await self.archive_data(data_type, cutoff_date) await self.delete_old_data(data_type, cutoff_date) await self.log_retention_action(data_type, cutoff_date) async def anonymize_old_data(self, data_type: str, before_date: datetime): """Anonymize data instead of deleting.""" # Example: Replace identifying fields with hashes if data_type == 'analytics_data': await self.db.analytics.update_many( {'collection_date': {'$lt': before_date}}, {'$set': { 'user_id': None, 'ip_address': None, 'device_id': None, 'anonymized': True, 'anonymized_date': datetime.utcnow() }} ) ``` ### Pattern 4: Privacy by Design ```python class PrivacyFirstDataModel: """Example of privacy-by-design data model.""" # Separate PII from behavioral data user_profile_schema = { 'user_id': str, # UUID, not sequential 'email_hash': str, # Hashed for lookups 'created_at': datetime, # Minimal data collection 'preferences': { 'language': str, 'timezone': str } } # Encrypted at rest user_pii_schema = { 'user_id': str, 'email': str, # Encrypted 'name': str, # Encrypted 'phone': str, # Encrypted (optional) 'address': dict, # Encrypted (optional) 'encryption_key_id': str } # Pseudonymized behavioral data analytics_schema = { 'session_id': str, # Not linked to user_id 'pseudonym_id': str, # Rotating pseudonym 'events': list, 'device_category': str, # Generalized, not specific 'country': str, # Not city-level } class DataMinimization: """Implement data minimization principles.""" @staticmethod def collect_only_needed(form_data: dict, purpose: str) -> dict: """Filter form data to only fields needed for purpose.""" REQUIRED_FIELDS = { 'account_creation': ['email', 'password'], 'newsletter': ['email'], 'purchase': ['email', 'name', 'address', 'payment'], 'support': ['email', 'message'] } allowed = REQUIRED_FIELDS.get(purpose, []) return {k: v for k, v in form_data.items() if k in allowed} @staticmethod def generalize_location(ip_address: str) -> str: """Generalize IP to country level only.""" import geoip2.database reader = geoip2.database.Reader('GeoLite2-Country.mmdb') try: response = reader.country(ip_address) return response.country.iso_code except: return 'UNKNOWN' ``` ### Pattern 5: Breach Notification ```python from datetime import datetime from enum import Enum class BreachSeverity(Enum): LOW = "low" MEDIUM = "medium" HIGH = "high" CRITICAL = "critical" class BreachNotificationHandler: """Handle GDPR breach notification requirements.""" AUTHORITY_NOTIFICATION_HOURS = 72 AFFECTED_NOTIFICATION_REQUIRED_SEVERITY = BreachSeverity.HIGH async def report_breach( self, description: str, data_types: List[str], affected_count: int, severity: BreachSeverity ) -> dict: """Report and handle a data breach.""" breach = { 'id': self.generate_breach_id(), 'reported_at': datetime.utcnow(), 'description': description, 'data_types_affected': data_types, 'affected_individuals_count': affected_count, 'severity': severity.value, 'status': 'investigating', 'timeline': [{ 'event': 'breach_reported', 'timestamp': datetime.utcnow(), 'details': description }] } await self.db.breaches.insert_one(breach) # Immediate notifications await self.notify_dpo(breach) await self.notify_security_team(breach) # Authority notification required within 72 hours if self.requires_authority_notification(severity, data_types): breach['authority_notification_deadline'] = ( datetime.utcnow() + timedelta(hours=self.AUTHORITY_NOTIFICATION_HOURS) ) await self.schedule_authority_notification(breach) # Affected individuals notification if severity.value in [BreachSeverity.HIGH.value, BreachSeverity.CRITICAL.value]: await self.schedule_individual_notifications(breach) return breach def requires_authority_notification( self, severity: BreachSeverity, data_types: List[str] ) -> bool: """Determine if supervisory authority must be notified.""" # Always notify for sensitive data sensitive_types = ['health', 'financial', 'credentials', 'biometric'] if any(t in sensitive_types for t in data_types): return True # Notify for medium+ severity return severity in [BreachSeverity.MEDIUM, BreachSeverity.HIGH, BreachSeverity.CRITICAL] async def generate_authority_report(self, breach_id: str) -> dict: """Generate report for supervisory authority.""" breach = await self.get_breach(breach_id) return { 'organization': { 'name': self.config.org_name, 'contact': self.config.dpo_contact, 'registration': self.config.registration_number }, 'breach': { 'nature': breach['description'], 'categories_affected': breach['data_types_affected'], 'approximate_number_affected': breach['affected_individuals_count'], 'likely_consequences': self.assess_consequences(breach), 'measures_taken': await self.get_remediation_measures(breach_id), 'measures_proposed': await self.get_proposed_measures(breach_id) }, 'timeline': breach['timeline'], 'submitted_at': datetime.utcnow().isoformat() } ``` ## Compliance Checklist ```markdown ## GDPR Implementation Checklist ### Legal Basis - [ ] Documented legal basis for each processing activity - [ ] Consent mechanisms meet GDPR requirements - [ ] Legitimate interest assessments completed ### Transparency - [ ] Privacy policy is clear and accessible - [ ] Processing purposes clearly stated - [ ] Data retention periods documented ### Data Subject Rights - [ ] Access request process implemented - [ ] Erasure request process implemented - [ ] Portability export available - [ ] Rectification process available - [ ] Response within 30-day deadline ### Security - [ ] Encryption at rest implemented - [ ] Encryption in transit (TLS) - [ ] Access controls in place - [ ] Audit logging enabled ### Breach Response - [ ] Breach detection mechanisms - [ ] 72-hour notification process - [ ] Breach documentation system ### Documentation - [ ] Records of processing activities (Art. 30) - [ ] Data protection impact assessments - [ ] Data processing agreements with vendors ``` ## Best Practices ### Do's - **Minimize data collection** - Only collect what's needed - **Document everything** - Processing activities, legal bases - **Encrypt PII** - At rest and in transit - **Implement access controls** - Need-to-know basis - **Regular audits** - Verify compliance continuously ### Don'ts - **Don't pre-check consent boxes** - Must be opt-in - **Don't bundle consent** - Separate purposes separately - **Don't retain indefinitely** - Defi
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

modern-javascript-patterns

Master ES6+ features including async/await, destructuring, spread

coding
⭐1
# Modern JavaScript Patterns Comprehensive guide for mastering modern JavaScript (ES6+) features, functional programming patterns, and best practices for writing clean, maintainable, and performant code. ## When to Use This Skill - Refactoring legacy JavaScript to modern syntax - Implementing functional programming patterns - Optimizing JavaScript performance - Writing maintainable and readable code - Working with asynchronous operations - Building modern web applications - Migrating from callbacks to Promises/async-await - Implementing data transformation pipelines ## ES6+ Core Features ### 1. Arrow Functions **Syntax and Use Cases:** ```javascript // Traditional function function add(a, b) { return a + b; } // Arrow function const add = (a, b) => a + b; // Single parameter (parentheses optional) const double = (x) => x * 2; // No parameters const getRandom = () => Math.random(); // Multiple statements (need curly braces) const processUser = (user) => { const normalized = user.name.toLowerCase(); return { ...user, name: normalized }; }; // Returning objects (wrap in parentheses) const createUser = (name, age) => ({ name, age }); ``` **Lexical 'this' Binding:** ```javascript class Counter { constructor() { this.count = 0; } // Arrow function preserves 'this' context increment = () => { this.count++; }; // Traditional function loses 'this' in callbacks incrementTraditional() { setTimeout(function () { this.count++; // 'this' is undefined }, 1000); } // Arrow function maintains 'this' incrementArrow() { setTimeout(() => { this.count++; // 'this' refers to Counter instance }, 1000); } } ``` ### 2. Destructuring **Object Destructuring:** ```javascript const user = { id: 1, name: "John Doe", email: "john@example.com", address: { city: "New York", country: "USA", }, }; // Basic destructuring const { name, email } = user; // Rename variables const { name: userName, email: userEmail } = user; // Default values const { age = 25 } = user; // Nested destructuring const { address: { city, country }, } = user; // Rest operator const { id, ...userWithoutId } = user; // Function parameters function greet({ name, age = 18 }) { console.log(`Hello ${name}, you are ${age}`); } greet(user); ``` **Array Destructuring:** ```javascript const numbers = [1, 2, 3, 4, 5]; // Basic destructuring const [first, second] = numbers; // Skip elements const [, , third] = numbers; // Rest operator const [head, ...tail] = numbers; // Swapping variables let a = 1, b = 2; [a, b] = [b, a]; // Function return values function getCoordinates() { return [10, 20]; } const [x, y] = getCoordinates(); // Default values const [one, two, three = 0] = [1, 2]; ``` ### 3. Spread and Rest Operators **Spread Operator:** ```javascript // Array spreading const arr1 = [1, 2, 3]; const arr2 = [4, 5, 6]; const combined = [...arr1, ...arr2]; // Object spreading const defaults = { theme: "dark", lang: "en" }; const userPrefs = { theme: "light" }; const settings = { ...defaults, ...userPrefs }; // Function arguments const numbers = [1, 2, 3]; Math.max(...numbers); // Copying arrays/objects (shallow copy) const copy = [...arr1]; const objCopy = { ...user }; // Adding items immutably const newArr = [...arr1, 4, 5]; const newObj = { ...user, age: 30 }; ``` **Rest Parameters:** ```javascript // Collect function arguments function sum(...numbers) { return numbers.reduce((total, num) => total + num, 0); } sum(1, 2, 3, 4, 5); // With regular parameters function greet(greeting, ...names) { return `${greeting} ${names.join(", ")}`; } greet("Hello", "John", "Jane", "Bob"); // Object rest const { id, ...userData } = user; // Array rest const [first, ...rest] = [1, 2, 3, 4, 5]; ``` ### 4. Template Literals ```javascript // Basic usage const name = "John"; const greeting = `Hello, ${name}!`; // Multi-line strings const html = ` <div> <h1>${title}</h1> <p>${content}</p> </div> `; // Expression evaluation const price = 19.99; const total = `Total: $${(price * 1.2).toFixed(2)}`; // Tagged template literals function highlight(strings, ...values) { return strings.reduce((result, str, i) => { const value = values[i] || ""; return result + str + `<mark>${value}</mark>`; }, ""); } const name = "John"; const age = 30; const html = highlight`Name: ${name}, Age: ${age}`; // Output: "Name: <mark>John</mark>, Age: <mark>30</mark>" ``` ### 5. Enhanced Object Literals ```javascript const name = "John"; const age = 30; // Shorthand property names const user = { name, age }; // Shorthand method names const calculator = { add(a, b) { return a + b; }, subtract(a, b) { return a - b; }, }; // Computed property names const field = "email"; const user = { name: "John", [field]: "john@example.com", [`get${field.charAt(0).toUpperCase()}${field.slice(1)}`]() { return this[field]; }, }; // Dynamic property creation const createUser = (name, ...props) => { return props.reduce( (user, [key, value]) => ({ ...user, [key]: value, }), { name }, ); }; const user = createUser("John", ["age", 30], ["email", "john@example.com"]); ``` ## Asynchronous Patterns ### 1. Promises **Creating and Using Promises:** ```javascript // Creating a promise const fetchUser = (id) => { return new Promise((resolve, reject) => { setTimeout(() => { if (id > 0) { resolve({ id, name: "John" }); } else { reject(new Error("Invalid ID")); } }, 1000); }); }; // Using promises fetchUser(1) .then((user) => console.log(user)) .catch((error) => console.error(error)) .finally(() => console.log("Done")); // Chaining promises fetchUser(1) .then((user) => fetchUserPosts(user.id)) .then((posts) => processPosts(posts)) .then((result) => console.log(result)) .catch((error) => console.error(error)); ``` **Promise Combinators:** ```javascript // Promise.all - Wait for all promises const promises = [fetchUser(1), fetchUser(2), fetchUser(3)]; Promise.all(promises) .then((users) => console.log(users)) .catch((error) => console.error("At least one failed:", error)); // Promise.allSettled - Wait for all, regardless of outcome Promise.allSettled(promises).then((results) => { results.forEach((result) => { if (result.status === "fulfilled") { console.log("Success:", result.value); } else { console.log("Error:", result.reason); } }); }); // Promise.race - First to complete Promise.race(promises) .then((winner) => console.log("First:", winner)) .catch((error) => console.error(error)); // Promise.any - First to succeed Promise.any(promises) .then((first) => console.log("First success:", first)) .catch((error) => console.error("All failed:", error)); ``` ### 2. Async/Await **Basic Usage:** ```javascript // Async function always returns a Promise async function fetchUser(id) { const response = await fetch(`/api/users/${id}`); const user = await response.json(); return user; } // Error handling with try/catch async function getUserData(id) { try { const user = await fetchUser(id); const posts = await fetchUserPosts(user.id); return { user, posts }; } catch (error) { console.error("Error fetching data:", error); throw error; } } // Sequential vs Parallel execution async function sequential() { const user1 = await fetchUser(1); // Wait const user2 = await fetchUser(2); // Then wait return [user1, user2]; } async function parallel() { const [user1, user2] = await Promise.all([fetchUser(1), fetchUser(2)]); return [user1, user2]; } ``` **Advanced Patterns:** ```javascript // Async IIFE (async () => { const result = await someAsyncOperation(); console.log(result); })(); // Async iteration async function processUsers(userIds) { for (const id of userIds) { const user = await fetchUser(id); await processUser(user); } } // Top-level await (ES2022) const config = await fetch("/config.json").then((r) => r.json()); // Retry logic async function fetchWithRetry(url, retries = 3) { for (let i = 0; i < retries; i++) { try { return await fetch(url); } catch (error) { if (i === retries - 1) throw error; await new Promise((resolve) => setTimeout(resolve, 1000 * (i + 1))); } } } // Timeout wrapper async function withTimeout(promise, ms) { const timeout = new Promise((_, reject) => setTimeout(() => reject(new Error("Timeout")), ms), ); return Promise.race([promise, timeout]); } ``` ## Functional Programming Patterns ### 1. Array Methods **Map, Filter, Reduce:** ```javascript const users = [ { id: 1, name: "John", age: 30, active: true }, { id: 2, name: "Jane", age: 25, active: false }, { id: 3, name: "Bob", age: 35, active: true }, ]; // Map - Transform array const names = users.map((user) => user.name); const upperNames = users.map((user) => user.name.toUpperCase()); // Filter - Select elements const activeUsers = users.filter((user) => user.active); const adults = users.filter((user) => user.age >= 18); // Reduce - Aggregate data const totalAge = users.reduce((sum, user) => sum + user.age, 0); const avgAge = totalAge / users.length; // Group by property const byActive = users.reduce((groups, user) => { const key = user.active ? "active" : "inactive"; return { ...groups, [key]: [...(groups[key] || []), user], }; }, {}); // Chaining methods const result = users .filter((user) => user.active) .map((user) => user.name) .sort() .join(", "); ``` **Advanced Array Methods:** ```javascript // Find - First matching element const user = users.find((u) => u.id === 2); // FindIndex - Index of first match const index = users.findIndex((u) => u.name === "Jane"); // Some - At least one matches const hasActive = users.some((u) => u.active); // Every - All match const allAdults = users.every((u) => u.age >= 18); // FlatMap - Map and flatten const userTags = [ { name: "John", tags: ["admin", "user"] }, { name: "Jane", tags: ["user"] }, ]; const allTags = userTags.flatMap((u) => u.tags); // From - Create array from iterable const str = "hello"; const chars = Array.from(str); const numbers = Array.from({ length: 5 }, (_, i) => i + 1); // Of - Create array from arguments const arr = Array.of(1, 2, 3); ``` ### 2. Higher-Order Functions **Functions as Arguments:** ```javascript // Custom forEach function forEach(array, callback) { for (let i = 0; i < array.length; i++) { callback(array[i], i, array); } } // Custom map function map(array, transform) { const result = []; for (const item of array) { result.push(transform(item)); } return result; } // Custom filter function filter(array, predicate) { const result = []; for (const item of array) { if (predicate(item)) { result.push(item); } } return result; } ``` **Functions Returning Functions:** ```javascript // Currying const multiply = (a) => (b) => a * b; const double = multiply(2); const triple = multiply(3); console.log(double(5)); // 10 console.log(triple(5)); // 15 // Partial application function partial(fn, ...args) { return (...moreArgs) => fn(...args, ...moreArgs); } const add = (a, b, c) => a + b + c; const add5 = partial(add, 5); console.log(add5(3, 2)); // 10 // Memoization function memoize(fn) { const cache = new Map(); return (...args) => { const key = JSON.stringify(args); if (cache.has(key)) { return cache.get(key); } const result = fn(...args); cache.set(key, result); return result; }; } const fibonacci = memoize((n) => { if (n <= 1) return n; return fibonacci(n - 1) + fibonacci(n - 2); }); ``` ### 3. Composition and Piping ```javascript // Function composition const compose = (...fns) => (x) => fns.reduceRight((acc, fn) => fn(acc), x); const pipe = (...fns) => (x) => fns.reduce((acc, fn) => fn(acc), x); // Example usage const addOne = (x) => x + 1; const double = (x) => x * 2; const square = (x) => x * x; const composed = compose(square, double, addOne); console.log(composed(3)); // ((3 + 1) * 2)^2 = 64 const piped = pipe(addOne, double, square); console.log(piped(3)); // ((3 + 1) * 2)^2 = 64 // Practical example const processUser = pipe( (user) => ({ ...user, name: user.name.trim() }), (user) => ({ ...user, email: user.email.toLowerCase() }), (user) => ({ ...user, age: parseInt(user.age) }), ); const user = processUser({ name: " John ", email: "JOHN@EXAMPLE.COM", age: "30", }); ``` ### 4. Pure Functions and Immutability ```javascript // Impure function (modifies input) function addItemImpure(cart, item) { cart.items.push(item); cart.total += item.price; return cart; } // Pure function (no side effects) function addItemPure(cart, item) { return { ...cart, items: [...cart.items, item], total: cart.total + item.price, }; } // Immutable array operations const numbers = [1, 2, 3, 4, 5]; // Add to array const withSix = [...numbers, 6]; // Remove from array const withoutThree = numbers.filter((n) => n !== 3); // Update array element const doubled = numbers.map((n) => (n === 3 ? n * 2 : n)); // Immutable object operations const user = { name: "John", age: 30 }; // Update property const olderUser = { ...user, age: 31 }; // Add property const withEmail = { ...user, email: "john@example.com" }; // Remove property const { age, ...withoutAge } = user; // Deep cloning (simple approach) const deepClone = (obj) => JSON.parse(JSON.stringify(obj)); // Better deep cloning const structuredClone = (obj) => globalThis.structuredClone(obj); ``` ## Modern Class Features ```javascript // Class syntax class User { // Private fields #password; // Public fields id; name; // Static field static count = 0; constructor(id, name, password) { this.id = id; this.name = name; this.#password = password; User.count++; } // Public method greet() { return `Hello, ${this.name}`; } // Private method #hashPassword(password) { return `hashed_${password}`; } // Getter get displayName() { return this.name.toUpperCase(); } // Setter set password(newPassword) { this.#password = this.#hashPassword(newPassword); } // Static method static create(id, name, password) { return new User(id, name, password); } } // Inheritance class Admin extends User { constructor(id, name, password, role) { super(id, name, password); this.role = role; } greet() { return `${super.greet()}, I'm an admin`; } } ``` ## Modules (ES6) ```javascript // Exporting // math.js export const PI = 3.14159; export function add(a, b) { return a + b; } export class Calculator { // ... } // Default export export default function multiply(a, b) { return a * b; } // Importing // app.js import multiply, { PI, add, Calculator } from "./math.js"; // Rename imports import { add as sum } from "./math.js"; // Import all import * as Math from "./math.js"; // Dynamic imports const module = await import("./math.js"); const { add } = await import("./math.js"); // Conditional loading if (condition) { const module = await import("./feature.js"); module.init(); } ``` ## Iterators and Generators ```javascript // Custom iterator const range = { from: 1, to: 5, [Symbol.iterator]() { return { current: this.from, last: this.to, next() { if (this.current <= this.last) { return { done: false, value: this.current++ }; } else { return { done: true }; } }, }; }, }; for (const num of range) { console.log(num); // 1, 2, 3, 4, 5 } // Generator function function* rangeGenerator(from, to) { for (let i = from; i <= to; i++) { yield i; } } for (const num of rangeGenerator(1, 5)) { console.log(num); } // Infinite generator function* fibonacci() { let [prev, curr] = [0, 1]; while (true) { yield curr; [prev, curr] = [curr, prev + curr]; } } // Async generator async function* fetchPages(url) { let page = 1; while (true) { const response = await fetch(`${url}?page=${page}`); const data = await response.json(); if (data.length === 0) break; yield data; page++; } } for await (const page of fetchPages("/api/users")) { console.log(page); } ``` ## Modern Operators ```javascript // Optional chaining const user = { name: "John", address: { city: "NYC" } }; const city = user?.address?.city; const zipCode = user?.address?.zipCode; // undefined // Function call const result = obj.method?.(); // Array access const first = arr?.[0]; // Nullish coalescing const value = null ?? "default"; // 'default' const value = undefined ?? "default"; // 'default' const value = 0 ?? "default"; // 0 (not 'default') const value = "" ?? "default"; // '' (not 'default') // Logical assignment let a = null; a ??= "default"; // a = 'default' let b = 5; b ??= 10; // b = 5 (unchanged) let obj = { count: 0 }; obj.count ||= 1; // obj.count = 1 obj.count &&= 2; // obj.count = 2 ``` ## Performance Optimization ```javascript // Debounce function debounce(fn, delay) { let timeoutId; return (...args) => { clearTimeout(timeoutId); timeoutId = setTimeout(() => fn(...args), delay); }; } const searchDebounced = debounce(search, 300); // Throttle function throttle(fn, limit) { let inThrottle; return (...args) => { if (!inThrottle) { fn(...args); inThrottle = true; setTimeout(() => (inThrottle = false), limit); } }; } const scrollThrottled = throttle(handleScroll, 100); // Lazy evaluation function* lazyMap(iterable, transform) { for (const item of iterable) { yield transform(item); } } // Use only what you need const numbers = [1, 2, 3, 4, 5]; const doubled = lazyMap(numbers, (x) => x * 2); const first = doubled.next().value; // Only computes first value ``` ## Best Practices 1. **Use const by default**: Only use let when reassignment is needed 2. **Prefer arrow functions**: Especially for callbacks 3. **Use template literals**: Instead of string concatenation 4. **Destructure objects and arrays**: For cleaner code 5. **Use async/await**: Instead of Promise chains 6. **Avoid mutating data**: Use spread operator and array methods 7. **Use optional chaining**: Prevent "Cannot read property of undefined" 8. **Use nullish coalescing**: For default values 9. **Prefer array methods**: Over traditional loops 10. **Use modules**: For better code organization 11. **Write pure functions**: Easier to test and reason about 12. **Use meaningful variable names**: Self-documenting code 13. **Keep functions small**: Single responsibility principle 14. **Handle errors properly**: Use try/catch with async/await 15. **Use strict mode**: `'use strict'` for better error catching ## Common Pitfalls 1. **this binding confusion**: Use arrow functions or bind() 2. **Async/await without error handling**: Always use try/catch 3. **Promise creation
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered
πŸ€–system promptβ€’7 months ago

team-composition-patterns

Design optimal agent team compositions with sizing heuristics,

coding
⭐1
# Team Composition Patterns Best practices for composing multi-agent teams, selecting team sizes, choosing agent types, and configuring display modes for Claude Code's Agent Teams feature. ## When to Use This Skill - Deciding how many teammates to spawn for a task - Choosing between preset team configurations - Selecting the right agent type (subagent_type) for each role - Configuring teammate display modes (tmux, iTerm2, in-process) - Building custom team compositions for non-standard workflows ## Team Sizing Heuristics | Complexity | Team Size | When to Use | | ------------ | --------- | ----------------------------------------------------------- | | Simple | 1-2 | Single-dimension review, isolated bug, small feature | | Moderate | 2-3 | Multi-file changes, 2-3 concerns, medium features | | Complex | 3-4 | Cross-cutting concerns, large features, deep debugging | | Very Complex | 4-5 | Full-stack features, comprehensive reviews, systemic issues | **Rule of thumb**: Start with the smallest team that covers all required dimensions. Adding teammates increases coordination overhead. ## Preset Team Compositions ### Review Team - **Size**: 3 reviewers - **Agents**: 3x `team-reviewer` - **Default dimensions**: security, performance, architecture - **Use when**: Code changes need multi-dimensional quality assessment ### Debug Team - **Size**: 3 investigators - **Agents**: 3x `team-debugger` - **Default hypotheses**: 3 competing hypotheses - **Use when**: Bug has multiple plausible root causes ### Feature Team - **Size**: 3 (1 lead + 2 implementers) - **Agents**: 1x `team-lead` + 2x `team-implementer` - **Use when**: Feature can be decomposed into parallel work streams ### Fullstack Team - **Size**: 4 (1 lead + 3 implementers) - **Agents**: 1x `team-lead` + 1x frontend `team-implementer` + 1x backend `team-implementer` + 1x test `team-implementer` - **Use when**: Feature spans frontend, backend, and test layers ### Research Team - **Size**: 3 researchers - **Agents**: 3x `general-purpose` - **Default areas**: Each assigned a different research question, module, or topic - **Capabilities**: Codebase search (Grep, Glob, Read), web search (WebSearch, WebFetch) - **Use when**: Need to understand a codebase, research libraries, compare approaches, or gather information from code and web sources in parallel ### Security Team - **Size**: 4 reviewers - **Agents**: 4x `team-reviewer` - **Default dimensions**: OWASP/vulnerabilities, auth/access control, dependencies/supply chain, secrets/configuration - **Use when**: Comprehensive security audit covering multiple attack surfaces ### Migration Team - **Size**: 4 (1 lead + 2 implementers + 1 reviewer) - **Agents**: 1x `team-lead` + 2x `team-implementer` + 1x `team-reviewer` - **Use when**: Large codebase migration (framework upgrade, language port, API version bump) requiring parallel work with correctness verification ## Agent Type Selection When spawning teammates with the Task tool, choose `subagent_type` based on what tools the teammate needs: | Agent Type | Tools Available | Use For | | ------------------------------ | ----------------------------------------- | ---------------------------------------------------------- | | `general-purpose` | All tools (Read, Write, Edit, Bash, etc.) | Implementation, debugging, any task requiring file changes | | `Explore` | Read-only tools (Read, Grep, Glob) | Research, code exploration, analysis | | `Plan` | Read-only tools | Architecture planning, task decomposition | | `agent-teams:team-reviewer` | All tools | Code review with structured findings | | `agent-teams:team-debugger` | All tools | Hypothesis-driven investigation | | `agent-teams:team-implementer` | All tools | Building features within file ownership boundaries | | `agent-teams:team-lead` | All tools | Team orchestration and coordination | **Key distinction**: Read-only agents (Explore, Plan) cannot modify files. Never assign implementation tasks to read-only agents. ## Display Mode Configuration Configure in `~/.claude/settings.json`: ```json { "teammateMode": "tmux" } ``` | Mode | Behavior | Best For | | -------------- | ------------------------------ | ------------------------------------------------- | | `"tmux"` | Each teammate in a tmux pane | Development workflows, monitoring multiple agents | | `"iterm2"` | Each teammate in an iTerm2 tab | macOS users who prefer iTerm2 | | `"in-process"` | All teammates in same process | Simple tasks, CI/CD environments | ## Custom Team Guidelines When building custom teams: 1. **Every team needs a coordinator** β€” Either designate a `team-lead` or have the user coordinate directly 2. **Match roles to agent types** β€” Use specialized agents (reviewer, debugger, implementer) when available 3. **Avoid duplicate roles** β€” Two agents doing the same thing wastes resources 4. **Define boundaries upfront** β€” Each teammate needs clear ownership of files or responsibilities 5. **Keep it small** β€” 2-4 teammates is the sweet spot; 5+ requires significant coordination overhead
πŸ‘0
πŸ‘οΈ0
πŸ€– Auto-discovered