Skip to main content
EVOKORE// BROWSE
>

./browse/prompts

23 NODES
🤖system prompt•7 months ago

solidity-security

Master smart contract security best practices to prevent common

security
⭐1
# Solidity Security Master smart contract security best practices, vulnerability prevention, and secure Solidity development patterns. ## When to Use This Skill - Writing secure smart contracts - Auditing existing contracts for vulnerabilities - Implementing secure DeFi protocols - Preventing reentrancy, overflow, and access control issues - Optimizing gas usage while maintaining security - Preparing contracts for professional audits - Understanding common attack vectors ## Critical Vulnerabilities ### 1. Reentrancy Attacker calls back into your contract before state is updated. **Vulnerable Code:** ```solidity // VULNERABLE TO REENTRANCY contract VulnerableBank { mapping(address => uint256) public balances; function withdraw() public { uint256 amount = balances[msg.sender]; // DANGER: External call before state update (bool success, ) = msg.sender.call{value: amount}(""); require(success); balances[msg.sender] = 0; // Too late! } } ``` **Secure Pattern (Checks-Effects-Interactions):** ```solidity contract SecureBank { mapping(address => uint256) public balances; function withdraw() public { uint256 amount = balances[msg.sender]; require(amount > 0, "Insufficient balance"); // EFFECTS: Update state BEFORE external call balances[msg.sender] = 0; // INTERACTIONS: External call last (bool success, ) = msg.sender.call{value: amount}(""); require(success, "Transfer failed"); } } ``` **Alternative: ReentrancyGuard** ```solidity import "@openzeppelin/contracts/security/ReentrancyGuard.sol"; contract SecureBank is ReentrancyGuard { mapping(address => uint256) public balances; function withdraw() public nonReentrant { uint256 amount = balances[msg.sender]; require(amount > 0, "Insufficient balance"); balances[msg.sender] = 0; (bool success, ) = msg.sender.call{value: amount}(""); require(success, "Transfer failed"); } } ``` ### 2. Integer Overflow/Underflow **Vulnerable Code (Solidity < 0.8.0):** ```solidity // VULNERABLE contract VulnerableToken { mapping(address => uint256) public balances; function transfer(address to, uint256 amount) public { // No overflow check - can wrap around balances[msg.sender] -= amount; // Can underflow! balances[to] += amount; // Can overflow! } } ``` **Secure Pattern (Solidity >= 0.8.0):** ```solidity // Solidity 0.8+ has built-in overflow/underflow checks contract SecureToken { mapping(address => uint256) public balances; function transfer(address to, uint256 amount) public { // Automatically reverts on overflow/underflow balances[msg.sender] -= amount; balances[to] += amount; } } ``` **For Solidity < 0.8.0, use SafeMath:** ```solidity import "@openzeppelin/contracts/utils/math/SafeMath.sol"; contract SecureToken { using SafeMath for uint256; mapping(address => uint256) public balances; function transfer(address to, uint256 amount) public { balances[msg.sender] = balances[msg.sender].sub(amount); balances[to] = balances[to].add(amount); } } ``` ### 3. Access Control **Vulnerable Code:** ```solidity // VULNERABLE: Anyone can call critical functions contract VulnerableContract { address public owner; function withdraw(uint256 amount) public { // No access control! payable(msg.sender).transfer(amount); } } ``` **Secure Pattern:** ```solidity import "@openzeppelin/contracts/access/Ownable.sol"; contract SecureContract is Ownable { function withdraw(uint256 amount) public onlyOwner { payable(owner()).transfer(amount); } } // Or implement custom role-based access contract RoleBasedContract { mapping(address => bool) public admins; modifier onlyAdmin() { require(admins[msg.sender], "Not an admin"); _; } function criticalFunction() public onlyAdmin { // Protected function } } ``` ### 4. Front-Running **Vulnerable:** ```solidity // VULNERABLE TO FRONT-RUNNING contract VulnerableDEX { function swap(uint256 amount, uint256 minOutput) public { // Attacker sees this in mempool and front-runs uint256 output = calculateOutput(amount); require(output >= minOutput, "Slippage too high"); // Perform swap } } ``` **Mitigation:** ```solidity contract SecureDEX { mapping(bytes32 => bool) public usedCommitments; // Step 1: Commit to trade function commitTrade(bytes32 commitment) public { usedCommitments[commitment] = true; } // Step 2: Reveal trade (next block) function revealTrade( uint256 amount, uint256 minOutput, bytes32 secret ) public { bytes32 commitment = keccak256(abi.encodePacked( msg.sender, amount, minOutput, secret )); require(usedCommitments[commitment], "Invalid commitment"); // Perform swap } } ``` ## Security Best Practices ### Checks-Effects-Interactions Pattern ```solidity contract SecurePattern { mapping(address => uint256) public balances; function withdraw(uint256 amount) public { // 1. CHECKS: Validate conditions require(amount <= balances[msg.sender], "Insufficient balance"); require(amount > 0, "Amount must be positive"); // 2. EFFECTS: Update state balances[msg.sender] -= amount; // 3. INTERACTIONS: External calls last (bool success, ) = msg.sender.call{value: amount}(""); require(success, "Transfer failed"); } } ``` ### Pull Over Push Pattern ```solidity // Prefer this (pull) contract SecurePayment { mapping(address => uint256) public pendingWithdrawals; function recordPayment(address recipient, uint256 amount) internal { pendingWithdrawals[recipient] += amount; } function withdraw() public { uint256 amount = pendingWithdrawals[msg.sender]; require(amount > 0, "Nothing to withdraw"); pendingWithdrawals[msg.sender] = 0; payable(msg.sender).transfer(amount); } } // Over this (push) contract RiskyPayment { function distributePayments(address[] memory recipients, uint256[] memory amounts) public { for (uint i = 0; i < recipients.length; i++) { // If any transfer fails, entire batch fails payable(recipients[i]).transfer(amounts[i]); } } } ``` ### Input Validation ```solidity contract SecureContract { function transfer(address to, uint256 amount) public { // Validate inputs require(to != address(0), "Invalid recipient"); require(to != address(this), "Cannot send to contract"); require(amount > 0, "Amount must be positive"); require(amount <= balances[msg.sender], "Insufficient balance"); // Proceed with transfer balances[msg.sender] -= amount; balances[to] += amount; } } ``` ### Emergency Stop (Circuit Breaker) ```solidity import "@openzeppelin/contracts/security/Pausable.sol"; contract EmergencyStop is Pausable, Ownable { function criticalFunction() public whenNotPaused { // Function logic } function emergencyStop() public onlyOwner { _pause(); } function resume() public onlyOwner { _unpause(); } } ``` ## Gas Optimization ### Use `uint256` Instead of Smaller Types ```solidity // More gas efficient contract GasEfficient { uint256 public value; // Optimal function set(uint256 _value) public { value = _value; } } // Less efficient contract GasInefficient { uint8 public value; // Still uses 256-bit slot function set(uint8 _value) public { value = _value; // Extra gas for type conversion } } ``` ### Pack Storage Variables ```solidity // Gas efficient (3 variables in 1 slot) contract PackedStorage { uint128 public a; // Slot 0 uint64 public b; // Slot 0 uint64 public c; // Slot 0 uint256 public d; // Slot 1 } // Gas inefficient (each variable in separate slot) contract UnpackedStorage { uint256 public a; // Slot 0 uint256 public b; // Slot 1 uint256 public c; // Slot 2 uint256 public d; // Slot 3 } ``` ### Use `calldata` Instead of `memory` for Function Arguments ```solidity contract GasOptimized { // More gas efficient function processData(uint256[] calldata data) public pure returns (uint256) { return data[0]; } // Less efficient function processDataMemory(uint256[] memory data) public pure returns (uint256) { return data[0]; } } ``` ### Use Events for Data Storage (When Appropriate) ```solidity contract EventStorage { // Emitting events is cheaper than storage event DataStored(address indexed user, uint256 indexed id, bytes data); function storeData(uint256 id, bytes calldata data) public { emit DataStored(msg.sender, id, data); // Don't store in contract storage unless needed } } ``` ## Common Vulnerabilities Checklist ```solidity // Security Checklist Contract contract SecurityChecklist { /** * [ ] Reentrancy protection (ReentrancyGuard or CEI pattern) * [ ] Integer overflow/underflow (Solidity 0.8+ or SafeMath) * [ ] Access control (Ownable, roles, modifiers) * [ ] Input validation (require statements) * [ ] Front-running mitigation (commit-reveal if applicable) * [ ] Gas optimization (packed storage, calldata) * [ ] Emergency stop mechanism (Pausable) * [ ] Pull over push pattern for payments * [ ] No delegatecall to untrusted contracts * [ ] No tx.origin for authentication (use msg.sender) * [ ] Proper event emission * [ ] External calls at end of function * [ ] Check return values of external calls * [ ] No hardcoded addresses * [ ] Upgrade mechanism (if proxy pattern) */ } ``` ## Testing for Security ```javascript // Hardhat test example const { expect } = require("chai"); const { ethers } = require("hardhat"); describe("Security Tests", function () { it("Should prevent reentrancy attack", async function () { const [attacker] = await ethers.getSigners(); const VictimBank = await ethers.getContractFactory("SecureBank"); const bank = await VictimBank.deploy(); const Attacker = await ethers.getContractFactory("ReentrancyAttacker"); const attackerContract = await Attacker.deploy(bank.address); // Deposit funds await bank.deposit({ value: ethers.utils.parseEther("10") }); // Attempt reentrancy attack await expect( attackerContract.attack({ value: ethers.utils.parseEther("1") }), ).to.be.revertedWith("ReentrancyGuard: reentrant call"); }); it("Should prevent integer overflow", async function () { const Token = await ethers.getContractFactory("SecureToken"); const token = await Token.deploy(); // Attempt overflow await expect(token.transfer(attacker.address, ethers.constants.MaxUint256)) .to.be.reverted; }); it("Should enforce access control", async function () { const [owner, attacker] = await ethers.getSigners(); const Contract = await ethers.getContractFactory("SecureContract"); const contract = await Contract.deploy(); // Attempt unauthorized withdrawal await expect(contract.connect(attacker).withdraw(100)).to.be.revertedWith( "Ownable: caller is not the owner", ); }); }); ``` ## Audit Preparation ```solidity contract WellDocumentedContract { /** * @title Well Documented Contract * @dev Example of proper documentation for audits * @notice This contract handles user deposits and withdrawals */ /// @notice Mapping of user balances mapping(address => uint256) public balances; /** * @dev Deposits ETH into the contract * @notice Anyone can deposit funds */ function deposit() public payable { require(msg.value > 0, "Must send ETH"); balances[msg.sender] += msg.value; } /** * @dev Withdraws user's balance * @notice Follows CEI pattern to prevent reentrancy * @param amount Amount to withdraw in wei */ function withdraw(uint256 amount) public { // CHECKS require(amount <= balances[msg.sender], "Insufficient balance"); // EFFECTS balances[msg.sender] -= amount; // INTERACTIONS (bool success, ) = msg.sender.call{value: amount}(""); require(success, "Transfer failed"); } } ``` ## Resources - **references/reentrancy.md**: Comprehensive reentrancy prevention - **references/access-control.md**: Role-based access patterns - **references/overflow-underflow.md**: SafeMath and integer safety - **references/gas-optimization.md**: Gas saving techniques - **references/vulnerability-patterns.md**: Common vulnerability catalog - **assets/solidity-contracts-templates.sol**: Secure contract templates - **assets/security-checklist.md**: Pre-audit checklist - **scripts/analyze-contract.sh**: Static analysis tools ## Tools for Security Analysis - **Slither**: Static analysis tool - **Mythril**: Security analysis tool - **Echidna**: Fuzzing tool - **Manticore**: Symbolic execution - **Securify**: Automated security scanner ## Common Pitfalls 1. **Using `tx.origin` for Authentication**: Use `msg.sender` instead 2. **Unchecked External Calls**: Always check return values 3. **Delegatecall to Untrusted Contracts**: Can hijack your contract 4. **Floating Pragma**: Pin to specific Solidity version 5. **Missing Events**: Emit events for state changes 6. **Excessive Gas in Loops**: Can hit block gas limit 7. **No Upgrade Path**: Consider proxy patterns if upgrades needed
👍0
👁️0
🤖 Auto-discovered
🤖system prompt•7 months ago

web3-testing

Test smart contracts comprehensively using Hardhat and Foundry with

coding
⭐1
# Web3 Smart Contract Testing Master comprehensive testing strategies for smart contracts using Hardhat, Foundry, and advanced testing patterns. ## When to Use This Skill - Writing unit tests for smart contracts - Setting up integration test suites - Performing gas optimization testing - Fuzzing for edge cases - Forking mainnet for realistic testing - Automating test coverage reporting - Verifying contracts on Etherscan ## Hardhat Testing Setup ```javascript // hardhat.config.js require("@nomicfoundation/hardhat-toolbox"); require("@nomiclabs/hardhat-etherscan"); require("hardhat-gas-reporter"); require("solidity-coverage"); module.exports = { solidity: { version: "0.8.19", settings: { optimizer: { enabled: true, runs: 200, }, }, }, networks: { hardhat: { forking: { url: process.env.MAINNET_RPC_URL, blockNumber: 15000000, }, }, goerli: { url: process.env.GOERLI_RPC_URL, accounts: [process.env.PRIVATE_KEY], }, }, gasReporter: { enabled: true, currency: "USD", coinmarketcap: process.env.COINMARKETCAP_API_KEY, }, etherscan: { apiKey: process.env.ETHERSCAN_API_KEY, }, }; ``` ## Unit Testing Patterns ```javascript const { expect } = require("chai"); const { ethers } = require("hardhat"); const { loadFixture, time, } = require("@nomicfoundation/hardhat-network-helpers"); describe("Token Contract", function () { // Fixture for test setup async function deployTokenFixture() { const [owner, addr1, addr2] = await ethers.getSigners(); const Token = await ethers.getContractFactory("Token"); const token = await Token.deploy(); return { token, owner, addr1, addr2 }; } describe("Deployment", function () { it("Should set the right owner", async function () { const { token, owner } = await loadFixture(deployTokenFixture); expect(await token.owner()).to.equal(owner.address); }); it("Should assign total supply to owner", async function () { const { token, owner } = await loadFixture(deployTokenFixture); const ownerBalance = await token.balanceOf(owner.address); expect(await token.totalSupply()).to.equal(ownerBalance); }); }); describe("Transactions", function () { it("Should transfer tokens between accounts", async function () { const { token, owner, addr1 } = await loadFixture(deployTokenFixture); await expect(token.transfer(addr1.address, 50)).to.changeTokenBalances( token, [owner, addr1], [-50, 50], ); }); it("Should fail if sender doesn't have enough tokens", async function () { const { token, addr1 } = await loadFixture(deployTokenFixture); const initialBalance = await token.balanceOf(addr1.address); await expect( token.connect(addr1).transfer(owner.address, 1), ).to.be.revertedWith("Insufficient balance"); }); it("Should emit Transfer event", async function () { const { token, owner, addr1 } = await loadFixture(deployTokenFixture); await expect(token.transfer(addr1.address, 50)) .to.emit(token, "Transfer") .withArgs(owner.address, addr1.address, 50); }); }); describe("Time-based tests", function () { it("Should handle time-locked operations", async function () { const { token } = await loadFixture(deployTokenFixture); // Increase time by 1 day await time.increase(86400); // Test time-dependent functionality }); }); describe("Gas optimization", function () { it("Should use gas efficiently", async function () { const { token } = await loadFixture(deployTokenFixture); const tx = await token.transfer(addr1.address, 100); const receipt = await tx.wait(); expect(receipt.gasUsed).to.be.lessThan(50000); }); }); }); ``` ## Foundry Testing (Forge) ```solidity // SPDX-License-Identifier: MIT pragma solidity ^0.8.0; import "forge-std/Test.sol"; import "../src/Token.sol"; contract TokenTest is Test { Token token; address owner = address(1); address user1 = address(2); address user2 = address(3); function setUp() public { vm.prank(owner); token = new Token(); } function testInitialSupply() public { assertEq(token.totalSupply(), 1000000 * 10**18); } function testTransfer() public { vm.prank(owner); token.transfer(user1, 100); assertEq(token.balanceOf(user1), 100); assertEq(token.balanceOf(owner), token.totalSupply() - 100); } function testFailTransferInsufficientBalance() public { vm.prank(user1); token.transfer(user2, 100); // Should fail } function testCannotTransferToZeroAddress() public { vm.prank(owner); vm.expectRevert("Invalid recipient"); token.transfer(address(0), 100); } // Fuzzing test function testFuzzTransfer(uint256 amount) public { vm.assume(amount > 0 && amount <= token.totalSupply()); vm.prank(owner); token.transfer(user1, amount); assertEq(token.balanceOf(user1), amount); } // Test with cheatcodes function testDealAndPrank() public { // Give ETH to address vm.deal(user1, 10 ether); // Impersonate address vm.prank(user1); // Test functionality assertEq(user1.balance, 10 ether); } // Mainnet fork test function testForkMainnet() public { vm.createSelectFork("https://eth-mainnet.alchemyapi.io/v2/..."); // Interact with mainnet contracts address dai = 0x6B175474E89094C44Da98b954EedeAC495271d0F; assertEq(IERC20(dai).symbol(), "DAI"); } } ``` ## Advanced Testing Patterns ### Snapshot and Revert ```javascript describe("Complex State Changes", function () { let snapshotId; beforeEach(async function () { snapshotId = await network.provider.send("evm_snapshot"); }); afterEach(async function () { await network.provider.send("evm_revert", [snapshotId]); }); it("Test 1", async function () { // Make state changes }); it("Test 2", async function () { // State reverted, clean slate }); }); ``` ### Mainnet Forking ```javascript describe("Mainnet Fork Tests", function () { let uniswapRouter, dai, usdc; before(async function () { await network.provider.request({ method: "hardhat_reset", params: [ { forking: { jsonRpcUrl: process.env.MAINNET_RPC_URL, blockNumber: 15000000, }, }, ], }); // Connect to existing mainnet contracts uniswapRouter = await ethers.getContractAt( "IUniswapV2Router", "0x7a250d5630B4cF539739dF2C5dAcb4c659F2488D", ); dai = await ethers.getContractAt( "IERC20", "0x6B175474E89094C44Da98b954EedeAC495271d0F", ); }); it("Should swap on Uniswap", async function () { // Test with real Uniswap contracts }); }); ``` ### Impersonating Accounts ```javascript it("Should impersonate whale account", async function () { const whaleAddress = "0x..."; await network.provider.request({ method: "hardhat_impersonateAccount", params: [whaleAddress], }); const whale = await ethers.getSigner(whaleAddress); // Use whale's tokens await dai .connect(whale) .transfer(addr1.address, ethers.utils.parseEther("1000")); }); ``` ## Gas Optimization Testing ```javascript const { expect } = require("chai"); describe("Gas Optimization", function () { it("Compare gas usage between implementations", async function () { const Implementation1 = await ethers.getContractFactory("OptimizedContract"); const Implementation2 = await ethers.getContractFactory( "UnoptimizedContract", ); const contract1 = await Implementation1.deploy(); const contract2 = await Implementation2.deploy(); const tx1 = await contract1.doSomething(); const receipt1 = await tx1.wait(); const tx2 = await contract2.doSomething(); const receipt2 = await tx2.wait(); console.log("Optimized gas:", receipt1.gasUsed.toString()); console.log("Unoptimized gas:", receipt2.gasUsed.toString()); expect(receipt1.gasUsed).to.be.lessThan(receipt2.gasUsed); }); }); ``` ## Coverage Reporting ```bash # Generate coverage report npx hardhat coverage # Output shows: # File | % Stmts | % Branch | % Funcs | % Lines | # -------------------|---------|----------|---------|---------| # contracts/Token.sol | 100 | 90 | 100 | 95 | ``` ## Contract Verification ```javascript // Verify on Etherscan await hre.run("verify:verify", { address: contractAddress, constructorArguments: [arg1, arg2], }); ``` ```bash # Or via CLI npx hardhat verify --network mainnet CONTRACT_ADDRESS "Constructor arg1" "arg2" ``` ## CI/CD Integration ```yaml # .github/workflows/test.yml name: Tests on: [push, pull_request] jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - uses: actions/setup-node@v2 with: node-version: "16" - run: npm install - run: npx hardhat compile - run: npx hardhat test - run: npx hardhat coverage - name: Upload coverage to Codecov uses: codecov/codecov-action@v2 ``` ## Resources - **references/hardhat-setup.md**: Hardhat configuration guide - **references/foundry-setup.md**: Foundry testing framework - **references/test-patterns.md**: Testing best practices - **references/mainnet-forking.md**: Fork testing strategies - **references/contract-verification.md**: Etherscan verification - **assets/hardhat-config.js**: Complete Hardhat configuration - **assets/test-suite.js**: Comprehensive test examples - **assets/foundry.toml**: Foundry configuration - **scripts/test-contract.sh**: Automated testing script ## Best Practices 1. **Test Coverage**: Aim for >90% coverage 2. **Edge Cases**: Test boundary conditions 3. **Gas Limits**: Verify functions don't hit block gas limit 4. **Reentrancy**: Test for reentrancy vulnerabilities 5. **Access Control**: Test unauthorized access attempts 6. **Events**: Verify event emissions 7. **Fixtures**: Use fixtures to avoid code duplication 8. **Mainnet Fork**: Test with real contracts 9. **Fuzzing**: Use property-based testing 10. **CI/CD**: Automate testing on every commit
👍0
👁️0
🤖 Auto-discovered
🤖system prompt•7 months ago

screen-reader-testing

Test web applications with screen readers including VoiceOver,

coding
⭐1
# Screen Reader Testing Practical guide to testing web applications with screen readers for comprehensive accessibility validation. ## When to Use This Skill - Validating screen reader compatibility - Testing ARIA implementations - Debugging assistive technology issues - Verifying form accessibility - Testing dynamic content announcements - Ensuring navigation accessibility ## Core Concepts ### 1. Major Screen Readers | Screen Reader | Platform | Browser | Usage | | ------------- | --------- | -------------- | ----- | | **VoiceOver** | macOS/iOS | Safari | ~15% | | **NVDA** | Windows | Firefox/Chrome | ~31% | | **JAWS** | Windows | Chrome/IE | ~40% | | **TalkBack** | Android | Chrome | ~10% | | **Narrator** | Windows | Edge | ~4% | ### 2. Testing Priority ``` Minimum Coverage: 1. NVDA + Firefox (Windows) 2. VoiceOver + Safari (macOS) 3. VoiceOver + Safari (iOS) Comprehensive Coverage: + JAWS + Chrome (Windows) + TalkBack + Chrome (Android) + Narrator + Edge (Windows) ``` ### 3. Screen Reader Modes | Mode | Purpose | When Used | | ------------------ | ---------------------- | ----------------- | | **Browse/Virtual** | Read content | Default reading | | **Focus/Forms** | Interact with controls | Filling forms | | **Application** | Custom widgets | ARIA applications | ## VoiceOver (macOS) ### Setup ``` Enable: System Preferences → Accessibility → VoiceOver Toggle: Cmd + F5 Quick Toggle: Triple-press Touch ID ``` ### Essential Commands ``` Navigation: VO = Ctrl + Option (VoiceOver modifier) VO + Right Arrow Next element VO + Left Arrow Previous element VO + Shift + Down Enter group VO + Shift + Up Exit group Reading: VO + A Read all from cursor Ctrl Stop speaking VO + B Read current paragraph Interaction: VO + Space Activate element VO + Shift + M Open menu Tab Next focusable element Shift + Tab Previous focusable element Rotor (VO + U): Navigate by: Headings, Links, Forms, Landmarks Left/Right Arrow Change rotor category Up/Down Arrow Navigate within category Enter Go to item Web Specific: VO + Cmd + H Next heading VO + Cmd + J Next form control VO + Cmd + L Next link VO + Cmd + T Next table ``` ### Testing Checklist ```markdown ## VoiceOver Testing Checklist ### Page Load - [ ] Page title announced - [ ] Main landmark found - [ ] Skip link works ### Navigation - [ ] All headings discoverable via rotor - [ ] Heading levels logical (H1 → H2 → H3) - [ ] Landmarks properly labeled - [ ] Skip links functional ### Links & Buttons - [ ] Link purpose clear - [ ] Button actions described - [ ] New window/tab announced ### Forms - [ ] All labels read with inputs - [ ] Required fields announced - [ ] Error messages read - [ ] Instructions available - [ ] Focus moves to errors ### Dynamic Content - [ ] Alerts announced immediately - [ ] Loading states communicated - [ ] Content updates announced - [ ] Modals trap focus correctly ### Tables - [ ] Headers associated with cells - [ ] Table navigation works - [ ] Complex tables have captions ``` ### Common Issues & Fixes ```html <!-- Issue: Button not announcing purpose --> <button><svg>...</svg></button> <!-- Fix --> <button aria-label="Close dialog"><svg aria-hidden="true">...</svg></button> <!-- Issue: Dynamic content not announced --> <div id="results">New results loaded</div> <!-- Fix --> <div id="results" role="status" aria-live="polite">New results loaded</div> <!-- Issue: Form error not read --> <input type="email" /> <span class="error">Invalid email</span> <!-- Fix --> <input type="email" aria-invalid="true" aria-describedby="email-error" /> <span id="email-error" role="alert">Invalid email</span> ``` ## NVDA (Windows) ### Setup ``` Download: nvaccess.org Start: Ctrl + Alt + N Stop: Insert + Q ``` ### Essential Commands ``` Navigation: Insert = NVDA modifier Down Arrow Next line Up Arrow Previous line Tab Next focusable Shift + Tab Previous focusable Reading: NVDA + Down Arrow Say all Ctrl Stop speech NVDA + Up Arrow Current line Headings: H Next heading Shift + H Previous heading 1-6 Heading level 1-6 Forms: F Next form field B Next button E Next edit field X Next checkbox C Next combo box Links: K Next link U Next unvisited link V Next visited link Landmarks: D Next landmark Shift + D Previous landmark Tables: T Next table Ctrl + Alt + Arrows Navigate cells Elements List (NVDA + F7): Shows all links, headings, form fields, landmarks ``` ### Browse vs Focus Mode ``` NVDA automatically switches modes: - Browse Mode: Arrow keys navigate content - Focus Mode: Arrow keys control interactive elements Manual switch: NVDA + Space Watch for: - "Browse mode" announcement when navigating - "Focus mode" when entering form fields - Application role forces forms mode ``` ### Testing Script ```markdown ## NVDA Test Script ### Initial Load 1. Navigate to page 2. Let page finish loading 3. Press Insert + Down to read all 4. Note: Page title, main content identified? ### Landmark Navigation 1. Press D repeatedly 2. Check: All main areas reachable? 3. Check: Landmarks properly labeled? ### Heading Navigation 1. Press Insert + F7 → Headings 2. Check: Logical heading structure? 3. Press H to navigate headings 4. Check: All sections discoverable? ### Form Testing 1. Press F to find first form field 2. Check: Label read? 3. Fill in invalid data 4. Submit form 5. Check: Errors announced? 6. Check: Focus moved to error? ### Interactive Elements 1. Tab through all interactive elements 2. Check: Each announces role and state 3. Activate buttons with Enter/Space 4. Check: Result announced? ### Dynamic Content 1. Trigger content update 2. Check: Change announced? 3. Open modal 4. Check: Focus trapped? 5. Close modal 6. Check: Focus returns? ``` ## JAWS (Windows) ### Essential Commands ``` Start: Desktop shortcut or Ctrl + Alt + J Virtual Cursor: Auto-enabled in browsers Navigation: Arrow keys Navigate content Tab Next focusable Insert + Down Read all Ctrl Stop speech Quick Keys: H Next heading T Next table F Next form field B Next button G Next graphic L Next list ; Next landmark Forms Mode: Enter Enter forms mode Numpad + Exit forms mode F5 List form fields Lists: Insert + F7 Link list Insert + F6 Heading list Insert + F5 Form field list Tables: Ctrl + Alt + Arrows Table navigation ``` ## TalkBack (Android) ### Setup ``` Enable: Settings → Accessibility → TalkBack Toggle: Hold both volume buttons 3 seconds ``` ### Gestures ``` Explore: Drag finger across screen Next: Swipe right Previous: Swipe left Activate: Double tap Scroll: Two finger swipe Reading Controls (swipe up then right): - Headings - Links - Controls - Characters - Words - Lines - Paragraphs ``` ## Common Test Scenarios ### 1. Modal Dialog ```html <!-- Accessible modal structure --> <div role="dialog" aria-modal="true" aria-labelledby="dialog-title" aria-describedby="dialog-desc" > <h2 id="dialog-title">Confirm Delete</h2> <p id="dialog-desc">This action cannot be undone.</p> <button>Cancel</button> <button>Delete</button> </div> ``` ```javascript // Focus management function openModal(modal) { // Store last focused element lastFocus = document.activeElement; // Move focus to modal modal.querySelector("h2").focus(); // Trap focus modal.addEventListener("keydown", trapFocus); } function closeModal(modal) { // Return focus lastFocus.focus(); } function trapFocus(e) { if (e.key === "Tab") { const focusable = modal.querySelectorAll( 'button, [href], input, select, textarea, [tabindex]:not([tabindex="-1"])', ); const first = focusable[0]; const last = focusable[focusable.length - 1]; if (e.shiftKey && document.activeElement === first) { last.focus(); e.preventDefault(); } else if (!e.shiftKey && document.activeElement === last) { first.focus(); e.preventDefault(); } } if (e.key === "Escape") { closeModal(modal); } } ``` ### 2. Live Regions ```html <!-- Status messages (polite) --> <div role="status" aria-live="polite" aria-atomic="true"> <!-- Content updates will be announced after current speech --> </div> <!-- Alerts (assertive) --> <div role="alert" aria-live="assertive"> <!-- Content updates interrupt current speech --> </div> <!-- Progress updates --> <div role="progressbar" aria-valuenow="75" aria-valuemin="0" aria-valuemax="100" aria-label="Upload progress" ></div> <!-- Log (additions only) --> <div role="log" aria-live="polite" aria-relevant="additions"> <!-- New messages announced, removals not --> </div> ``` ### 3. Tab Interface ```html <div role="tablist" aria-label="Product information"> <button role="tab" id="tab-1" aria-selected="true" aria-controls="panel-1"> Description </button> <button role="tab" id="tab-2" aria-selected="false" aria-controls="panel-2" tabindex="-1" > Reviews </button> </div> <div role="tabpanel" id="panel-1" aria-labelledby="tab-1"> Product description content... </div> <div role="tabpanel" id="panel-2" aria-labelledby="tab-2" hidden> Reviews content... </div> ``` ```javascript // Tab keyboard navigation tablist.addEventListener("keydown", (e) => { const tabs = [...tablist.querySelectorAll('[role="tab"]')]; const index = tabs.indexOf(document.activeElement); let newIndex; switch (e.key) { case "ArrowRight": newIndex = (index + 1) % tabs.length; break; case "ArrowLeft": newIndex = (index - 1 + tabs.length) % tabs.length; break; case "Home": newIndex = 0; break; case "End": newIndex = tabs.length - 1; break; default: return; } tabs[newIndex].focus(); activateTab(tabs[newIndex]); e.preventDefault(); }); ``` ## Debugging Tips ```javascript // Log what screen reader sees function logAccessibleName(element) { const computed = window.getComputedStyle(element); console.log({ role: element.getAttribute("role") || element.tagName, name: element.getAttribute("aria-label") || element.getAttribute("aria-labelledby") || element.textContent, state: { expanded: element.getAttribute("aria-expanded"), selected: element.getAttribute("aria-selected"), checked: element.getAttribute("aria-checked"), disabled: element.disabled, }, visible: computed.display !== "none" && computed.visibility !== "hidden", }); } ``` ## Best Practices ### Do's - **Test with actual screen readers** - Not just simulators - **Use semantic HTML first** - ARIA is supplemental - **Test in browse and focus modes** - Different experiences - **Verify focus management** - Especially for SPAs - **Test keyboard only first** - Foundation for SR testing ### Don'ts - **Don't assume one SR is enough** - Test multiple - **Don't ignore mobile** - Growing user base - **Don't test only happy path** - Test error states - **Don't skip dynamic content** - Most common issues - **Don't rely on visual testing** - Different experience ## Resources - [VoiceOver User Guide](https://support.apple.com/guide/voiceover/welcome/mac) - [NVDA User Guide](https://www.nvaccess.org/files/nvda/documentation/userGuide.html) - [JAWS Documentation](https://support.freedomscientific.com/Products/Blindness/JAWS) - [WebAIM Screen Reader Survey](https://webaim.org/projects/screenreadersurvey/)
👍1
👁️1
🤖 Auto-discovered
🤖system prompt•7 months ago

auth-implementation-patterns

Master authentication and authorization patterns including JWT,

coding
⭐1
# Authentication & Authorization Implementation Patterns Build secure, scalable authentication and authorization systems using industry-standard patterns and modern best practices. ## When to Use This Skill - Implementing user authentication systems - Securing REST or GraphQL APIs - Adding OAuth2/social login - Implementing role-based access control (RBAC) - Designing session management - Migrating authentication systems - Debugging auth issues - Implementing SSO or multi-tenancy ## Core Concepts ### 1. Authentication vs Authorization **Authentication (AuthN)**: Who are you? - Verifying identity (username/password, OAuth, biometrics) - Issuing credentials (sessions, tokens) - Managing login/logout **Authorization (AuthZ)**: What can you do? - Permission checking - Role-based access control (RBAC) - Resource ownership validation - Policy enforcement ### 2. Authentication Strategies **Session-Based:** - Server stores session state - Session ID in cookie - Traditional, simple, stateful **Token-Based (JWT):** - Stateless, self-contained - Scales horizontally - Can store claims **OAuth2/OpenID Connect:** - Delegate authentication - Social login (Google, GitHub) - Enterprise SSO ## JWT Authentication ### Pattern 1: JWT Implementation ```typescript // JWT structure: header.payload.signature import jwt from "jsonwebtoken"; import { Request, Response, NextFunction } from "express"; interface JWTPayload { userId: string; email: string; role: string; iat: number; exp: number; } // Generate JWT function generateTokens(userId: string, email: string, role: string) { const accessToken = jwt.sign( { userId, email, role }, process.env.JWT_SECRET!, { expiresIn: "15m" }, // Short-lived ); const refreshToken = jwt.sign( { userId }, process.env.JWT_REFRESH_SECRET!, { expiresIn: "7d" }, // Long-lived ); return { accessToken, refreshToken }; } // Verify JWT function verifyToken(token: string): JWTPayload { try { return jwt.verify(token, process.env.JWT_SECRET!) as JWTPayload; } catch (error) { if (error instanceof jwt.TokenExpiredError) { throw new Error("Token expired"); } if (error instanceof jwt.JsonWebTokenError) { throw new Error("Invalid token"); } throw error; } } // Middleware function authenticate(req: Request, res: Response, next: NextFunction) { const authHeader = req.headers.authorization; if (!authHeader?.startsWith("Bearer ")) { return res.status(401).json({ error: "No token provided" }); } const token = authHeader.substring(7); try { const payload = verifyToken(token); req.user = payload; // Attach user to request next(); } catch (error) { return res.status(401).json({ error: "Invalid token" }); } } // Usage app.get("/api/profile", authenticate, (req, res) => { res.json({ user: req.user }); }); ``` ### Pattern 2: Refresh Token Flow ```typescript interface StoredRefreshToken { token: string; userId: string; expiresAt: Date; createdAt: Date; } class RefreshTokenService { // Store refresh token in database async storeRefreshToken(userId: string, refreshToken: string) { const expiresAt = new Date(Date.now() + 7 * 24 * 60 * 60 * 1000); await db.refreshTokens.create({ token: await hash(refreshToken), // Hash before storing userId, expiresAt, }); } // Refresh access token async refreshAccessToken(refreshToken: string) { // Verify refresh token let payload; try { payload = jwt.verify(refreshToken, process.env.JWT_REFRESH_SECRET!) as { userId: string; }; } catch { throw new Error("Invalid refresh token"); } // Check if token exists in database const storedToken = await db.refreshTokens.findOne({ where: { token: await hash(refreshToken), userId: payload.userId, expiresAt: { $gt: new Date() }, }, }); if (!storedToken) { throw new Error("Refresh token not found or expired"); } // Get user const user = await db.users.findById(payload.userId); if (!user) { throw new Error("User not found"); } // Generate new access token const accessToken = jwt.sign( { userId: user.id, email: user.email, role: user.role }, process.env.JWT_SECRET!, { expiresIn: "15m" }, ); return { accessToken }; } // Revoke refresh token (logout) async revokeRefreshToken(refreshToken: string) { await db.refreshTokens.deleteOne({ token: await hash(refreshToken), }); } // Revoke all user tokens (logout all devices) async revokeAllUserTokens(userId: string) { await db.refreshTokens.deleteMany({ userId }); } } // API endpoints app.post("/api/auth/refresh", async (req, res) => { const { refreshToken } = req.body; try { const { accessToken } = await refreshTokenService.refreshAccessToken(refreshToken); res.json({ accessToken }); } catch (error) { res.status(401).json({ error: "Invalid refresh token" }); } }); app.post("/api/auth/logout", authenticate, async (req, res) => { const { refreshToken } = req.body; await refreshTokenService.revokeRefreshToken(refreshToken); res.json({ message: "Logged out successfully" }); }); ``` ## Session-Based Authentication ### Pattern 1: Express Session ```typescript import session from "express-session"; import RedisStore from "connect-redis"; import { createClient } from "redis"; // Setup Redis for session storage const redisClient = createClient({ url: process.env.REDIS_URL, }); await redisClient.connect(); app.use( session({ store: new RedisStore({ client: redisClient }), secret: process.env.SESSION_SECRET!, resave: false, saveUninitialized: false, cookie: { secure: process.env.NODE_ENV === "production", // HTTPS only httpOnly: true, // No JavaScript access maxAge: 24 * 60 * 60 * 1000, // 24 hours sameSite: "strict", // CSRF protection }, }), ); // Login app.post("/api/auth/login", async (req, res) => { const { email, password } = req.body; const user = await db.users.findOne({ email }); if (!user || !(await verifyPassword(password, user.passwordHash))) { return res.status(401).json({ error: "Invalid credentials" }); } // Store user in session req.session.userId = user.id; req.session.role = user.role; res.json({ user: { id: user.id, email: user.email, role: user.role } }); }); // Session middleware function requireAuth(req: Request, res: Response, next: NextFunction) { if (!req.session.userId) { return res.status(401).json({ error: "Not authenticated" }); } next(); } // Protected route app.get("/api/profile", requireAuth, async (req, res) => { const user = await db.users.findById(req.session.userId); res.json({ user }); }); // Logout app.post("/api/auth/logout", (req, res) => { req.session.destroy((err) => { if (err) { return res.status(500).json({ error: "Logout failed" }); } res.clearCookie("connect.sid"); res.json({ message: "Logged out successfully" }); }); }); ``` ## OAuth2 / Social Login ### Pattern 1: OAuth2 with Passport.js ```typescript import passport from "passport"; import { Strategy as GoogleStrategy } from "passport-google-oauth20"; import { Strategy as GitHubStrategy } from "passport-github2"; // Google OAuth passport.use( new GoogleStrategy( { clientID: process.env.GOOGLE_CLIENT_ID!, clientSecret: process.env.GOOGLE_CLIENT_SECRET!, callbackURL: "/api/auth/google/callback", }, async (accessToken, refreshToken, profile, done) => { try { // Find or create user let user = await db.users.findOne({ googleId: profile.id, }); if (!user) { user = await db.users.create({ googleId: profile.id, email: profile.emails?.[0]?.value, name: profile.displayName, avatar: profile.photos?.[0]?.value, }); } return done(null, user); } catch (error) { return done(error, undefined); } }, ), ); // Routes app.get( "/api/auth/google", passport.authenticate("google", { scope: ["profile", "email"], }), ); app.get( "/api/auth/google/callback", passport.authenticate("google", { session: false }), (req, res) => { // Generate JWT const tokens = generateTokens(req.user.id, req.user.email, req.user.role); // Redirect to frontend with token res.redirect( `${process.env.FRONTEND_URL}/auth/callback?token=${tokens.accessToken}`, ); }, ); ``` ## Authorization Patterns ### Pattern 1: Role-Based Access Control (RBAC) ```typescript enum Role { USER = "user", MODERATOR = "moderator", ADMIN = "admin", } const roleHierarchy: Record<Role, Role[]> = { [Role.ADMIN]: [Role.ADMIN, Role.MODERATOR, Role.USER], [Role.MODERATOR]: [Role.MODERATOR, Role.USER], [Role.USER]: [Role.USER], }; function hasRole(userRole: Role, requiredRole: Role): boolean { return roleHierarchy[userRole].includes(requiredRole); } // Middleware function requireRole(...roles: Role[]) { return (req: Request, res: Response, next: NextFunction) => { if (!req.user) { return res.status(401).json({ error: "Not authenticated" }); } if (!roles.some((role) => hasRole(req.user.role, role))) { return res.status(403).json({ error: "Insufficient permissions" }); } next(); }; } // Usage app.delete( "/api/users/:id", authenticate, requireRole(Role.ADMIN), async (req, res) => { // Only admins can delete users await db.users.delete(req.params.id); res.json({ message: "User deleted" }); }, ); ``` ### Pattern 2: Permission-Based Access Control ```typescript enum Permission { READ_USERS = "read:users", WRITE_USERS = "write:users", DELETE_USERS = "delete:users", READ_POSTS = "read:posts", WRITE_POSTS = "write:posts", } const rolePermissions: Record<Role, Permission[]> = { [Role.USER]: [Permission.READ_POSTS, Permission.WRITE_POSTS], [Role.MODERATOR]: [ Permission.READ_POSTS, Permission.WRITE_POSTS, Permission.READ_USERS, ], [Role.ADMIN]: Object.values(Permission), }; function hasPermission(userRole: Role, permission: Permission): boolean { return rolePermissions[userRole]?.includes(permission) ?? false; } function requirePermission(...permissions: Permission[]) { return (req: Request, res: Response, next: NextFunction) => { if (!req.user) { return res.status(401).json({ error: "Not authenticated" }); } const hasAllPermissions = permissions.every((permission) => hasPermission(req.user.role, permission), ); if (!hasAllPermissions) { return res.status(403).json({ error: "Insufficient permissions" }); } next(); }; } // Usage app.get( "/api/users", authenticate, requirePermission(Permission.READ_USERS), async (req, res) => { const users = await db.users.findAll(); res.json({ users }); }, ); ``` ### Pattern 3: Resource Ownership ```typescript // Check if user owns resource async function requireOwnership( resourceType: "post" | "comment", resourceIdParam: string = "id", ) { return async (req: Request, res: Response, next: NextFunction) => { if (!req.user) { return res.status(401).json({ error: "Not authenticated" }); } const resourceId = req.params[resourceIdParam]; // Admins can access anything if (req.user.role === Role.ADMIN) { return next(); } // Check ownership let resource; if (resourceType === "post") { resource = await db.posts.findById(resourceId); } else if (resourceType === "comment") { resource = await db.comments.findById(resourceId); } if (!resource) { return res.status(404).json({ error: "Resource not found" }); } if (resource.userId !== req.user.userId) { return res.status(403).json({ error: "Not authorized" }); } next(); }; } // Usage app.put( "/api/posts/:id", authenticate, requireOwnership("post"), async (req, res) => { // User can only update their own posts const post = await db.posts.update(req.params.id, req.body); res.json({ post }); }, ); ``` ## Security Best Practices ### Pattern 1: Password Security ```typescript import bcrypt from "bcrypt"; import { z } from "zod"; // Password validation schema const passwordSchema = z .string() .min(12, "Password must be at least 12 characters") .regex(/[A-Z]/, "Password must contain uppercase letter") .regex(/[a-z]/, "Password must contain lowercase letter") .regex(/[0-9]/, "Password must contain number") .regex(/[^A-Za-z0-9]/, "Password must contain special character"); // Hash password async function hashPassword(password: string): Promise<string> { const saltRounds = 12; // 2^12 iterations return bcrypt.hash(password, saltRounds); } // Verify password async function verifyPassword( password: string, hash: string, ): Promise<boolean> { return bcrypt.compare(password, hash); } // Registration with password validation app.post("/api/auth/register", async (req, res) => { try { const { email, password } = req.body; // Validate password passwordSchema.parse(password); // Check if user exists const existingUser = await db.users.findOne({ email }); if (existingUser) { return res.status(400).json({ error: "Email already registered" }); } // Hash password const passwordHash = await hashPassword(password); // Create user const user = await db.users.create({ email, passwordHash, }); // Generate tokens const tokens = generateTokens(user.id, user.email, user.role); res.status(201).json({ user: { id: user.id, email: user.email }, ...tokens, }); } catch (error) { if (error instanceof z.ZodError) { return res.status(400).json({ error: error.errors[0].message }); } res.status(500).json({ error: "Registration failed" }); } }); ``` ### Pattern 2: Rate Limiting ```typescript import rateLimit from "express-rate-limit"; import RedisStore from "rate-limit-redis"; // Login rate limiter const loginLimiter = rateLimit({ store: new RedisStore({ client: redisClient }), windowMs: 15 * 60 * 1000, // 15 minutes max: 5, // 5 attempts message: "Too many login attempts, please try again later", standardHeaders: true, legacyHeaders: false, }); // API rate limiter const apiLimiter = rateLimit({ windowMs: 60 * 1000, // 1 minute max: 100, // 100 requests per minute standardHeaders: true, }); // Apply to routes app.post("/api/auth/login", loginLimiter, async (req, res) => { // Login logic }); app.use("/api/", apiLimiter); ``` ## Best Practices 1. **Never Store Plain Passwords**: Always hash with bcrypt/argon2 2. **Use HTTPS**: Encrypt data in transit 3. **Short-Lived Access Tokens**: 15-30 minutes max 4. **Secure Cookies**: httpOnly, secure, sameSite flags 5. **Validate All Input**: Email format, password strength 6. **Rate Limit Auth Endpoints**: Prevent brute force attacks 7. **Implement CSRF Protection**: For session-based auth 8. **Rotate Secrets Regularly**: JWT secrets, session secrets 9. **Log Security Events**: Login attempts, failed auth 10. **Use MFA When Possible**: Extra security layer ## Common Pitfalls - **Weak Passwords**: Enforce strong password policies - **JWT in localStorage**: Vulnerable to XSS, use httpOnly cookies - **No Token Expiration**: Tokens should expire - **Client-Side Auth Checks Only**: Always validate server-side - **Insecure Password Reset**: Use secure tokens with expiration - **No Rate Limiting**: Vulnerable to brute force - **Trusting Client Data**: Always validate on server ## Resources - **references/jwt-best-practices.md**: JWT implementation guide - **references/oauth2-flows.md**: OAuth2 flow diagrams and examples - **references/session-security.md**: Secure session management - **assets/auth-security-checklist.md**: Security review checklist - **assets/password-policy-template.md**: Password requirements template - **scripts/token-validator.ts**: JWT validation utility
👍0
👁️0
🤖 Auto-discovered
🤖system prompt•7 months ago

bazel-build-optimization

Optimize Bazel builds for large-scale monorepos. Use when

coding
⭐1
# Bazel Build Optimization Production patterns for Bazel in large-scale monorepos. ## When to Use This Skill - Setting up Bazel for monorepos - Configuring remote caching/execution - Optimizing build times - Writing custom Bazel rules - Debugging build issues - Migrating to Bazel ## Core Concepts ### 1. Bazel Architecture ``` workspace/ ├── WORKSPACE.bazel # External dependencies ├── .bazelrc # Build configurations ├── .bazelversion # Bazel version ├── BUILD.bazel # Root build file ├── apps/ │ └── web/ │ └── BUILD.bazel ├── libs/ │ └── utils/ │ └── BUILD.bazel └── tools/ └── bazel/ └── rules/ ``` ### 2. Key Concepts | Concept | Description | | ----------- | -------------------------------------- | | **Target** | Buildable unit (library, binary, test) | | **Package** | Directory with BUILD file | | **Label** | Target identifier `//path/to:target` | | **Rule** | Defines how to build a target | | **Aspect** | Cross-cutting build behavior | ## Templates ### Template 1: WORKSPACE Configuration ```python # WORKSPACE.bazel workspace(name = "myproject") load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") # Rules for JavaScript/TypeScript http_archive( name = "aspect_rules_js", sha256 = "...", strip_prefix = "rules_js-1.34.0", url = "https://github.com/aspect-build/rules_js/releases/download/v1.34.0/rules_js-v1.34.0.tar.gz", ) load("@aspect_rules_js//js:repositories.bzl", "rules_js_dependencies") rules_js_dependencies() load("@rules_nodejs//nodejs:repositories.bzl", "nodejs_register_toolchains") nodejs_register_toolchains( name = "nodejs", node_version = "20.9.0", ) load("@aspect_rules_js//npm:repositories.bzl", "npm_translate_lock") npm_translate_lock( name = "npm", pnpm_lock = "//:pnpm-lock.yaml", verify_node_modules_ignored = "//:.bazelignore", ) load("@npm//:repositories.bzl", "npm_repositories") npm_repositories() # Rules for Python http_archive( name = "rules_python", sha256 = "...", strip_prefix = "rules_python-0.27.0", url = "https://github.com/bazelbuild/rules_python/releases/download/0.27.0/rules_python-0.27.0.tar.gz", ) load("@rules_python//python:repositories.bzl", "py_repositories") py_repositories() ``` ### Template 2: .bazelrc Configuration ```bash # .bazelrc # Build settings build --enable_platform_specific_config build --incompatible_enable_cc_toolchain_resolution build --experimental_strict_conflict_checks # Performance build --jobs=auto build --local_cpu_resources=HOST_CPUS*.75 build --local_ram_resources=HOST_RAM*.75 # Caching build --disk_cache=~/.cache/bazel-disk build --repository_cache=~/.cache/bazel-repo # Remote caching (optional) build:remote-cache --remote_cache=grpcs://cache.example.com build:remote-cache --remote_upload_local_results=true build:remote-cache --remote_timeout=3600 # Remote execution (optional) build:remote-exec --remote_executor=grpcs://remote.example.com build:remote-exec --remote_instance_name=projects/myproject/instances/default build:remote-exec --jobs=500 # Platform configurations build:linux --platforms=//platforms:linux_x86_64 build:macos --platforms=//platforms:macos_arm64 # CI configuration build:ci --config=remote-cache build:ci --build_metadata=ROLE=CI build:ci --bes_results_url=https://results.example.com/invocation/ build:ci --bes_backend=grpcs://bes.example.com # Test settings test --test_output=errors test --test_summary=detailed # Coverage coverage --combined_report=lcov coverage --instrumentation_filter="//..." # Convenience aliases build:opt --compilation_mode=opt build:dbg --compilation_mode=dbg # Import user settings try-import %workspace%/user.bazelrc ``` ### Template 3: TypeScript Library BUILD ```python # libs/utils/BUILD.bazel load("@aspect_rules_ts//ts:defs.bzl", "ts_project") load("@aspect_rules_js//js:defs.bzl", "js_library") load("@npm//:defs.bzl", "npm_link_all_packages") npm_link_all_packages(name = "node_modules") ts_project( name = "utils_ts", srcs = glob(["src/**/*.ts"]), declaration = True, source_map = True, tsconfig = "//:tsconfig.json", deps = [ ":node_modules/@types/node", ], ) js_library( name = "utils", srcs = [":utils_ts"], visibility = ["//visibility:public"], ) # Tests load("@aspect_rules_jest//jest:defs.bzl", "jest_test") jest_test( name = "utils_test", config = "//:jest.config.js", data = [ ":utils", "//:node_modules/jest", ], node_modules = "//:node_modules", ) ``` ### Template 4: Python Library BUILD ```python # libs/ml/BUILD.bazel load("@rules_python//python:defs.bzl", "py_library", "py_test", "py_binary") load("@pip//:requirements.bzl", "requirement") py_library( name = "ml", srcs = glob(["src/**/*.py"]), deps = [ requirement("numpy"), requirement("pandas"), requirement("scikit-learn"), "//libs/utils:utils_py", ], visibility = ["//visibility:public"], ) py_test( name = "ml_test", srcs = glob(["tests/**/*.py"]), deps = [ ":ml", requirement("pytest"), ], size = "medium", timeout = "moderate", ) py_binary( name = "train", srcs = ["train.py"], deps = [":ml"], data = ["//data:training_data"], ) ``` ### Template 5: Custom Rule for Docker ```python # tools/bazel/rules/docker.bzl def _docker_image_impl(ctx): dockerfile = ctx.file.dockerfile base_image = ctx.attr.base_image layers = ctx.files.layers # Build the image output = ctx.actions.declare_file(ctx.attr.name + ".tar") args = ctx.actions.args() args.add("--dockerfile", dockerfile) args.add("--output", output) args.add("--base", base_image) args.add_all("--layer", layers) ctx.actions.run( inputs = [dockerfile] + layers, outputs = [output], executable = ctx.executable._builder, arguments = [args], mnemonic = "DockerBuild", progress_message = "Building Docker image %s" % ctx.label, ) return [DefaultInfo(files = depset([output]))] docker_image = rule( implementation = _docker_image_impl, attrs = { "dockerfile": attr.label( allow_single_file = [".dockerfile", "Dockerfile"], mandatory = True, ), "base_image": attr.string(mandatory = True), "layers": attr.label_list(allow_files = True), "_builder": attr.label( default = "//tools/docker:builder", executable = True, cfg = "exec", ), }, ) ``` ### Template 6: Query and Dependency Analysis ```bash # Find all dependencies of a target bazel query "deps(//apps/web:web)" # Find reverse dependencies (what depends on this) bazel query "rdeps(//..., //libs/utils:utils)" # Find all targets in a package bazel query "//libs/..." # Find changed targets since commit bazel query "rdeps(//..., set($(git diff --name-only HEAD~1 | sed 's/.*/"&"/' | tr '\n' ' ')))" # Generate dependency graph bazel query "deps(//apps/web:web)" --output=graph | dot -Tpng > deps.png # Find all test targets bazel query "kind('.*_test', //...)" # Find targets with specific tag bazel query "attr(tags, 'integration', //...)" # Compute build graph size bazel query "deps(//...)" --output=package | wc -l ``` ### Template 7: Remote Execution Setup ```python # platforms/BUILD.bazel platform( name = "linux_x86_64", constraint_values = [ "@platforms//os:linux", "@platforms//cpu:x86_64", ], exec_properties = { "container-image": "docker://gcr.io/myproject/bazel-worker:latest", "OSFamily": "Linux", }, ) platform( name = "remote_linux", parents = [":linux_x86_64"], exec_properties = { "Pool": "default", "dockerNetwork": "standard", }, ) # toolchains/BUILD.bazel toolchain( name = "cc_toolchain_linux", exec_compatible_with = [ "@platforms//os:linux", "@platforms//cpu:x86_64", ], target_compatible_with = [ "@platforms//os:linux", "@platforms//cpu:x86_64", ], toolchain = "@remotejdk11_linux//:jdk", toolchain_type = "@bazel_tools//tools/jdk:runtime_toolchain_type", ) ``` ## Performance Optimization ```bash # Profile build bazel build //... --profile=profile.json bazel analyze-profile profile.json # Identify slow actions bazel build //... --execution_log_json_file=exec_log.json # Memory profiling bazel build //... --memory_profile=memory.json # Skip analysis cache bazel build //... --notrack_incremental_state ``` ## Best Practices ### Do's - **Use fine-grained targets** - Better caching - **Pin dependencies** - Reproducible builds - **Enable remote caching** - Share build artifacts - **Use visibility wisely** - Enforce architecture - **Write BUILD files per directory** - Standard convention ### Don'ts - **Don't use glob for deps** - Explicit is better - **Don't commit bazel-\* dirs** - Add to .gitignore - **Don't skip WORKSPACE setup** - Foundation of build - **Don't ignore build warnings** - Technical debt ## Resources - [Bazel Documentation](https://bazel.build/docs) - [Bazel Remote Execution](https://bazel.build/docs/remote-execution) - [rules_js](https://github.com/aspect-build/rules_js)
👍0
👁️0
🤖 Auto-discovered
🤖system prompt•7 months ago

debugging-strategies

Master systematic debugging techniques, profiling tools, and root

coding
⭐1
# Debugging Strategies Transform debugging from frustrating guesswork into systematic problem-solving with proven strategies, powerful tools, and methodical approaches. ## When to Use This Skill - Tracking down elusive bugs - Investigating performance issues - Understanding unfamiliar codebases - Debugging production issues - Analyzing crash dumps and stack traces - Profiling application performance - Investigating memory leaks - Debugging distributed systems ## Core Principles ### 1. The Scientific Method **1. Observe**: What's the actual behavior? **2. Hypothesize**: What could be causing it? **3. Experiment**: Test your hypothesis **4. Analyze**: Did it prove/disprove your theory? **5. Repeat**: Until you find the root cause ### 2. Debugging Mindset **Don't Assume:** - "It can't be X" - Yes it can - "I didn't change Y" - Check anyway - "It works on my machine" - Find out why **Do:** - Reproduce consistently - Isolate the problem - Keep detailed notes - Question everything - Take breaks when stuck ### 3. Rubber Duck Debugging Explain your code and problem out loud (to a rubber duck, colleague, or yourself). Often reveals the issue. ## Systematic Debugging Process ### Phase 1: Reproduce ```markdown ## Reproduction Checklist 1. **Can you reproduce it?** - Always? Sometimes? Randomly? - Specific conditions needed? - Can others reproduce it? 2. **Create minimal reproduction** - Simplify to smallest example - Remove unrelated code - Isolate the problem 3. **Document steps** - Write down exact steps - Note environment details - Capture error messages ``` ### Phase 2: Gather Information ```markdown ## Information Collection 1. **Error Messages** - Full stack trace - Error codes - Console/log output 2. **Environment** - OS version - Language/runtime version - Dependencies versions - Environment variables 3. **Recent Changes** - Git history - Deployment timeline - Configuration changes 4. **Scope** - Affects all users or specific ones? - All browsers or specific ones? - Production only or also dev? ``` ### Phase 3: Form Hypothesis ```markdown ## Hypothesis Formation Based on gathered info, ask: 1. **What changed?** - Recent code changes - Dependency updates - Infrastructure changes 2. **What's different?** - Working vs broken environment - Working vs broken user - Before vs after 3. **Where could this fail?** - Input validation - Business logic - Data layer - External services ``` ### Phase 4: Test & Verify ```markdown ## Testing Strategies 1. **Binary Search** - Comment out half the code - Narrow down problematic section - Repeat until found 2. **Add Logging** - Strategic console.log/print - Track variable values - Trace execution flow 3. **Isolate Components** - Test each piece separately - Mock dependencies - Remove complexity 4. **Compare Working vs Broken** - Diff configurations - Diff environments - Diff data ``` ## Debugging Tools ### JavaScript/TypeScript Debugging ```typescript // Chrome DevTools Debugger function processOrder(order: Order) { debugger; // Execution pauses here const total = calculateTotal(order); console.log("Total:", total); // Conditional breakpoint if (order.items.length > 10) { debugger; // Only breaks if condition true } return total; } // Console debugging techniques console.log("Value:", value); // Basic console.table(arrayOfObjects); // Table format console.time("operation"); /* code */ console.timeEnd("operation"); // Timing console.trace(); // Stack trace console.assert(value > 0, "Value must be positive"); // Assertion // Performance profiling performance.mark("start-operation"); // ... operation code performance.mark("end-operation"); performance.measure("operation", "start-operation", "end-operation"); console.log(performance.getEntriesByType("measure")); ``` **VS Code Debugger Configuration:** ```json // .vscode/launch.json { "version": "0.2.0", "configurations": [ { "type": "node", "request": "launch", "name": "Debug Program", "program": "${workspaceFolder}/src/index.ts", "preLaunchTask": "tsc: build - tsconfig.json", "outFiles": ["${workspaceFolder}/dist/**/*.js"], "skipFiles": ["<node_internals>/**"] }, { "type": "node", "request": "launch", "name": "Debug Tests", "program": "${workspaceFolder}/node_modules/jest/bin/jest", "args": ["--runInBand", "--no-cache"], "console": "integratedTerminal" } ] } ``` ### Python Debugging ```python # Built-in debugger (pdb) import pdb def calculate_total(items): total = 0 pdb.set_trace() # Debugger starts here for item in items: total += item.price * item.quantity return total # Breakpoint (Python 3.7+) def process_order(order): breakpoint() # More convenient than pdb.set_trace() # ... code # Post-mortem debugging try: risky_operation() except Exception: import pdb pdb.post_mortem() # Debug at exception point # IPython debugging (ipdb) from ipdb import set_trace set_trace() # Better interface than pdb # Logging for debugging import logging logging.basicConfig(level=logging.DEBUG) logger = logging.getLogger(__name__) def fetch_user(user_id): logger.debug(f'Fetching user: {user_id}') user = db.query(User).get(user_id) logger.debug(f'Found user: {user}') return user # Profile performance import cProfile import pstats cProfile.run('slow_function()', 'profile_stats') stats = pstats.Stats('profile_stats') stats.sort_stats('cumulative') stats.print_stats(10) # Top 10 slowest ``` ### Go Debugging ```go // Delve debugger // Install: go install github.com/go-delve/delve/cmd/dlv@latest // Run: dlv debug main.go import ( "fmt" "runtime" "runtime/debug" ) // Print stack trace func debugStack() { debug.PrintStack() } // Panic recovery with debugging func processRequest() { defer func() { if r := recover(); r != nil { fmt.Println("Panic:", r) debug.PrintStack() } }() // ... code that might panic } // Memory profiling import _ "net/http/pprof" // Visit http://localhost:6060/debug/pprof/ // CPU profiling import ( "os" "runtime/pprof" ) f, _ := os.Create("cpu.prof") pprof.StartCPUProfile(f) defer pprof.StopCPUProfile() // ... code to profile ``` ## Advanced Debugging Techniques ### Technique 1: Binary Search Debugging ```bash # Git bisect for finding regression git bisect start git bisect bad # Current commit is bad git bisect good v1.0.0 # v1.0.0 was good # Git checks out middle commit # Test it, then: git bisect good # if it works git bisect bad # if it's broken # Continue until bug found git bisect reset # when done ``` ### Technique 2: Differential Debugging Compare working vs broken: ```markdown ## What's Different? | Aspect | Working | Broken | | ------------ | ----------- | -------------- | | Environment | Development | Production | | Node version | 18.16.0 | 18.15.0 | | Data | Empty DB | 1M records | | User | Admin | Regular user | | Browser | Chrome | Safari | | Time | During day | After midnight | Hypothesis: Time-based issue? Check timezone handling. ``` ### Technique 3: Trace Debugging ```typescript // Function call tracing function trace( target: any, propertyKey: string, descriptor: PropertyDescriptor, ) { const originalMethod = descriptor.value; descriptor.value = function (...args: any[]) { console.log(`Calling ${propertyKey} with args:`, args); const result = originalMethod.apply(this, args); console.log(`${propertyKey} returned:`, result); return result; }; return descriptor; } class OrderService { @trace calculateTotal(items: Item[]): number { return items.reduce((sum, item) => sum + item.price, 0); } } ``` ### Technique 4: Memory Leak Detection ```typescript // Chrome DevTools Memory Profiler // 1. Take heap snapshot // 2. Perform action // 3. Take another snapshot // 4. Compare snapshots // Node.js memory debugging if (process.memoryUsage().heapUsed > 500 * 1024 * 1024) { console.warn("High memory usage:", process.memoryUsage()); // Generate heap dump require("v8").writeHeapSnapshot(); } // Find memory leaks in tests let beforeMemory: number; beforeEach(() => { beforeMemory = process.memoryUsage().heapUsed; }); afterEach(() => { const afterMemory = process.memoryUsage().heapUsed; const diff = afterMemory - beforeMemory; if (diff > 10 * 1024 * 1024) { // 10MB threshold console.warn(`Possible memory leak: ${diff / 1024 / 1024}MB`); } }); ``` ## Debugging Patterns by Issue Type ### Pattern 1: Intermittent Bugs ```markdown ## Strategies for Flaky Bugs 1. **Add extensive logging** - Log timing information - Log all state transitions - Log external interactions 2. **Look for race conditions** - Concurrent access to shared state - Async operations completing out of order - Missing synchronization 3. **Check timing dependencies** - setTimeout/setInterval - Promise resolution order - Animation frame timing 4. **Stress test** - Run many times - Vary timing - Simulate load ``` ### Pattern 2: Performance Issues ```markdown ## Performance Debugging 1. **Profile first** - Don't optimize blindly - Measure before and after - Find bottlenecks 2. **Common culprits** - N+1 queries - Unnecessary re-renders - Large data processing - Synchronous I/O 3. **Tools** - Browser DevTools Performance tab - Lighthouse - Python: cProfile, line_profiler - Node: clinic.js, 0x ``` ### Pattern 3: Production Bugs ```markdown ## Production Debugging 1. **Gather evidence** - Error tracking (Sentry, Bugsnag) - Application logs - User reports - Metrics/monitoring 2. **Reproduce locally** - Use production data (anonymized) - Match environment - Follow exact steps 3. **Safe investigation** - Don't change production - Use feature flags - Add monitoring/logging - Test fixes in staging ``` ## Best Practices 1. **Reproduce First**: Can't fix what you can't reproduce 2. **Isolate the Problem**: Remove complexity until minimal case 3. **Read Error Messages**: They're usually helpful 4. **Check Recent Changes**: Most bugs are recent 5. **Use Version Control**: Git bisect, blame, history 6. **Take Breaks**: Fresh eyes see better 7. **Document Findings**: Help future you 8. **Fix Root Cause**: Not just symptoms ## Common Debugging Mistakes - **Making Multiple Changes**: Change one thing at a time - **Not Reading Error Messages**: Read the full stack trace - **Assuming It's Complex**: Often it's simple - **Debug Logging in Prod**: Remove before shipping - **Not Using Debugger**: console.log isn't always best - **Giving Up Too Soon**: Persistence pays off - **Not Testing the Fix**: Verify it actually works ## Quick Debugging Checklist ```markdown ## When Stuck, Check: - [ ] Spelling errors (typos in variable names) - [ ] Case sensitivity (fileName vs filename) - [ ] Null/undefined values - [ ] Array index off-by-one - [ ] Async timing (race conditions) - [ ] Scope issues (closure, hoisting) - [ ] Type mismatches - [ ] Missing dependencies - [ ] Environment variables - [ ] File paths (absolute vs relative) - [ ] Cache issues (clear cache) - [ ] Stale data (refresh database) ``` ## Resources - **references/debugging-tools-guide.md**: Comprehensive tool documentation - **references/performance-profiling.md**: Performance debugging guide - **references/production-debugging.md**: Debugging live systems - **assets/debugging-checklist.md**: Quick reference checklist - **assets/common-bugs.md**: Common bug patterns - **scripts/debug-helper.ts**: Debugging utility functions
👍0
👁️0
🤖 Auto-discovered
🤖system prompt•7 months ago

monorepo-management

Master monorepo management with Turborepo, Nx, and pnpm workspaces

coding
⭐1
# Monorepo Management Build efficient, scalable monorepos that enable code sharing, consistent tooling, and atomic changes across multiple packages and applications. ## When to Use This Skill - Setting up new monorepo projects - Migrating from multi-repo to monorepo - Optimizing build and test performance - Managing shared dependencies - Implementing code sharing strategies - Setting up CI/CD for monorepos - Versioning and publishing packages - Debugging monorepo-specific issues ## Core Concepts ### 1. Why Monorepos? **Advantages:** - Shared code and dependencies - Atomic commits across projects - Consistent tooling and standards - Easier refactoring - Simplified dependency management - Better code visibility **Challenges:** - Build performance at scale - CI/CD complexity - Access control - Large Git repository ### 2. Monorepo Tools **Package Managers:** - pnpm workspaces (recommended) - npm workspaces - Yarn workspaces **Build Systems:** - Turborepo (recommended for most) - Nx (feature-rich, complex) - Lerna (older, maintenance mode) ## Turborepo Setup ### Initial Setup ```bash # Create new monorepo npx create-turbo@latest my-monorepo cd my-monorepo # Structure: # apps/ # web/ - Next.js app # docs/ - Documentation site # packages/ # ui/ - Shared UI components # config/ - Shared configurations # tsconfig/ - Shared TypeScript configs # turbo.json - Turborepo configuration # package.json - Root package.json ``` ### Configuration ```json // turbo.json { "$schema": "https://turbo.build/schema.json", "globalDependencies": ["**/.env.*local"], "pipeline": { "build": { "dependsOn": ["^build"], "outputs": ["dist/**", ".next/**", "!.next/cache/**"] }, "test": { "dependsOn": ["build"], "outputs": ["coverage/**"] }, "lint": { "outputs": [] }, "dev": { "cache": false, "persistent": true }, "type-check": { "dependsOn": ["^build"], "outputs": [] } } } ``` ```json // package.json (root) { "name": "my-monorepo", "private": true, "workspaces": ["apps/*", "packages/*"], "scripts": { "build": "turbo run build", "dev": "turbo run dev", "test": "turbo run test", "lint": "turbo run lint", "format": "prettier --write \"**/*.{ts,tsx,md}\"", "clean": "turbo run clean && rm -rf node_modules" }, "devDependencies": { "turbo": "^1.10.0", "prettier": "^3.0.0", "typescript": "^5.0.0" }, "packageManager": "pnpm@8.0.0" } ``` ### Package Structure ```json // packages/ui/package.json { "name": "@repo/ui", "version": "0.0.0", "private": true, "main": "./dist/index.js", "types": "./dist/index.d.ts", "exports": { ".": { "import": "./dist/index.js", "types": "./dist/index.d.ts" }, "./button": { "import": "./dist/button.js", "types": "./dist/button.d.ts" } }, "scripts": { "build": "tsup src/index.ts --format esm,cjs --dts", "dev": "tsup src/index.ts --format esm,cjs --dts --watch", "lint": "eslint src/", "type-check": "tsc --noEmit" }, "devDependencies": { "@repo/tsconfig": "workspace:*", "tsup": "^7.0.0", "typescript": "^5.0.0" }, "dependencies": { "react": "^18.2.0" } } ``` ## pnpm Workspaces ### Setup ```yaml # pnpm-workspace.yaml packages: - "apps/*" - "packages/*" - "tools/*" ``` ```json // .npmrc # Hoist shared dependencies shamefully-hoist=true # Strict peer dependencies auto-install-peers=true strict-peer-dependencies=true # Performance store-dir=~/.pnpm-store ``` ### Dependency Management ```bash # Install dependency in specific package pnpm add react --filter @repo/ui pnpm add -D typescript --filter @repo/ui # Install workspace dependency pnpm add @repo/ui --filter web # Install in all packages pnpm add -D eslint -w # Update all dependencies pnpm update -r # Remove dependency pnpm remove react --filter @repo/ui ``` ### Scripts ```bash # Run script in specific package pnpm --filter web dev pnpm --filter @repo/ui build # Run in all packages pnpm -r build pnpm -r test # Run in parallel pnpm -r --parallel dev # Filter by pattern pnpm --filter "@repo/*" build pnpm --filter "...web" build # Build web and dependencies ``` ## Nx Monorepo ### Setup ```bash # Create Nx monorepo npx create-nx-workspace@latest my-org # Generate applications nx generate @nx/react:app my-app nx generate @nx/next:app my-next-app # Generate libraries nx generate @nx/react:lib ui-components nx generate @nx/js:lib utils ``` ### Configuration ```json // nx.json { "extends": "nx/presets/npm.json", "$schema": "./node_modules/nx/schemas/nx-schema.json", "targetDefaults": { "build": { "dependsOn": ["^build"], "inputs": ["production", "^production"], "cache": true }, "test": { "inputs": ["default", "^production", "{workspaceRoot}/jest.preset.js"], "cache": true }, "lint": { "inputs": ["default", "{workspaceRoot}/.eslintrc.json"], "cache": true } }, "namedInputs": { "default": ["{projectRoot}/**/*", "sharedGlobals"], "production": [ "default", "!{projectRoot}/**/?(*.)+(spec|test).[jt]s?(x)?(.snap)", "!{projectRoot}/tsconfig.spec.json" ], "sharedGlobals": [] } } ``` ### Running Tasks ```bash # Run task for specific project nx build my-app nx test ui-components nx lint utils # Run for affected projects nx affected:build nx affected:test --base=main # Visualize dependencies nx graph # Run in parallel nx run-many --target=build --all --parallel=3 ``` ## Shared Configurations ### TypeScript Configuration ```json // packages/tsconfig/base.json { "compilerOptions": { "strict": true, "esModuleInterop": true, "skipLibCheck": true, "forceConsistentCasingInFileNames": true, "module": "ESNext", "moduleResolution": "bundler", "resolveJsonModule": true, "isolatedModules": true, "incremental": true, "declaration": true }, "exclude": ["node_modules"] } // packages/tsconfig/react.json { "extends": "./base.json", "compilerOptions": { "jsx": "react-jsx", "lib": ["ES2022", "DOM", "DOM.Iterable"] } } // apps/web/tsconfig.json { "extends": "@repo/tsconfig/react.json", "compilerOptions": { "outDir": "dist", "rootDir": "src" }, "include": ["src"], "exclude": ["node_modules", "dist"] } ``` ### ESLint Configuration ```javascript // packages/config/eslint-preset.js module.exports = { extends: [ "eslint:recommended", "plugin:@typescript-eslint/recommended", "plugin:react/recommended", "plugin:react-hooks/recommended", "prettier", ], plugins: ["@typescript-eslint", "react", "react-hooks"], parser: "@typescript-eslint/parser", parserOptions: { ecmaVersion: 2022, sourceType: "module", ecmaFeatures: { jsx: true, }, }, settings: { react: { version: "detect", }, }, rules: { "@typescript-eslint/no-unused-vars": "error", "react/react-in-jsx-scope": "off", }, }; // apps/web/.eslintrc.js module.exports = { extends: ["@repo/config/eslint-preset"], rules: { // App-specific rules }, }; ``` ## Code Sharing Patterns ### Pattern 1: Shared UI Components ```typescript // packages/ui/src/button.tsx import * as React from 'react'; export interface ButtonProps { variant?: 'primary' | 'secondary'; children: React.ReactNode; onClick?: () => void; } export function Button({ variant = 'primary', children, onClick }: ButtonProps) { return ( <button className={`btn btn-${variant}`} onClick={onClick} > {children} </button> ); } // packages/ui/src/index.ts export { Button, type ButtonProps } from './button'; export { Input, type InputProps } from './input'; // apps/web/src/app.tsx import { Button } from '@repo/ui'; export function App() { return <Button variant="primary">Click me</Button>; } ``` ### Pattern 2: Shared Utilities ```typescript // packages/utils/src/string.ts export function capitalize(str: string): string { return str.charAt(0).toUpperCase() + str.slice(1); } export function truncate(str: string, length: number): string { return str.length > length ? str.slice(0, length) + "..." : str; } // packages/utils/src/index.ts export * from "./string"; export * from "./array"; export * from "./date"; // Usage in apps import { capitalize, truncate } from "@repo/utils"; ``` ### Pattern 3: Shared Types ```typescript // packages/types/src/user.ts export interface User { id: string; email: string; name: string; role: "admin" | "user"; } export interface CreateUserInput { email: string; name: string; password: string; } // Used in both frontend and backend import type { User, CreateUserInput } from "@repo/types"; ``` ## Build Optimization ### Turborepo Caching ```json // turbo.json { "pipeline": { "build": { // Build depends on dependencies being built first "dependsOn": ["^build"], // Cache these outputs "outputs": ["dist/**", ".next/**"], // Cache based on these inputs (default: all files) "inputs": ["src/**/*.tsx", "src/**/*.ts", "package.json"] }, "test": { // Run tests in parallel, don't depend on build "cache": true, "outputs": ["coverage/**"] } } } ``` ### Remote Caching ```bash # Turborepo Remote Cache (Vercel) npx turbo login npx turbo link # Custom remote cache # turbo.json { "remoteCache": { "signature": true, "enabled": true } } ``` ## CI/CD for Monorepos ### GitHub Actions ```yaml # .github/workflows/ci.yml name: CI on: push: branches: [main] pull_request: branches: [main] jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 with: fetch-depth: 0 # For Nx affected commands - uses: pnpm/action-setup@v2 with: version: 8 - uses: actions/setup-node@v3 with: node-version: 18 cache: "pnpm" - name: Install dependencies run: pnpm install --frozen-lockfile - name: Build run: pnpm turbo run build - name: Test run: pnpm turbo run test - name: Lint run: pnpm turbo run lint - name: Type check run: pnpm turbo run type-check ``` ### Deploy Affected Only ```yaml # Deploy only changed apps - name: Deploy affected apps run: | if pnpm nx affected:apps --base=origin/main --head=HEAD | grep -q "web"; then echo "Deploying web app" pnpm --filter web deploy fi ``` ## Best Practices 1. **Consistent Versioning**: Lock dependency versions across workspace 2. **Shared Configs**: Centralize ESLint, TypeScript, Prettier configs 3. **Dependency Graph**: Keep it acyclic, avoid circular dependencies 4. **Cache Effectively**: Configure inputs/outputs correctly 5. **Type Safety**: Share types between frontend/backend 6. **Testing Strategy**: Unit tests in packages, E2E in apps 7. **Documentation**: README in each package 8. **Release Strategy**: Use changesets for versioning ## Common Pitfalls - **Circular Dependencies**: A depends on B, B depends on A - **Phantom Dependencies**: Using deps not in package.json - **Incorrect Cache Inputs**: Missing files in Turborepo inputs - **Over-Sharing**: Sharing code that should be separate - **Under-Sharing**: Duplicating code across packages - **Large Monorepos**: Without proper tooling, builds slow down ## Publishing Packages ```bash # Using Changesets pnpm add -Dw @changesets/cli pnpm changeset init # Create changeset pnpm changeset # Version packages pnpm changeset version # Publish pnpm changeset publish ``` ```yaml # .github/workflows/release.yml - name: Create Release Pull Request or Publish uses: changesets/action@v1 with: publish: pnpm release env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} NPM_TOKEN: ${{ secrets.NPM_TOKEN }} ``` ## Resources - **references/turborepo-guide.md**: Comprehensive Turborepo documentation - **references/nx-guide.md**: Nx monorepo patterns - **references/pnpm-workspaces.md**: pnpm workspace features - **assets/monorepo-checklist.md**: Setup checklist - **assets/migration-guide.md**: Multi-repo to monorepo migration - **scripts/dependency-graph.ts**: Visualize package dependencies
👍0
👁️0
🤖 Auto-discovered
🤖system prompt•7 months ago

changelog-automation

Automate changelog generation from commits, PRs, and releases

coding
⭐1
# Changelog Automation Patterns and tools for automating changelog generation, release notes, and version management following industry standards. ## When to Use This Skill - Setting up automated changelog generation - Implementing Conventional Commits - Creating release note workflows - Standardizing commit message formats - Generating GitHub/GitLab release notes - Managing semantic versioning ## Core Concepts ### 1. Keep a Changelog Format ```markdown # Changelog All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [Unreleased] ### Added - New feature X ## [1.2.0] - 2024-01-15 ### Added - User profile avatars - Dark mode support ### Changed - Improved loading performance by 40% ### Deprecated - Old authentication API (use v2) ### Removed - Legacy payment gateway ### Fixed - Login timeout issue (#123) ### Security - Updated dependencies for CVE-2024-1234 [Unreleased]: https://github.com/user/repo/compare/v1.2.0...HEAD [1.2.0]: https://github.com/user/repo/compare/v1.1.0...v1.2.0 ``` ### 2. Conventional Commits ``` <type>[optional scope]: <description> [optional body] [optional footer(s)] ``` | Type | Description | Changelog Section | | ---------- | ---------------- | ------------------ | | `feat` | New feature | Added | | `fix` | Bug fix | Fixed | | `docs` | Documentation | (usually excluded) | | `style` | Formatting | (usually excluded) | | `refactor` | Code restructure | Changed | | `perf` | Performance | Changed | | `test` | Tests | (usually excluded) | | `chore` | Maintenance | (usually excluded) | | `ci` | CI changes | (usually excluded) | | `build` | Build system | (usually excluded) | | `revert` | Revert commit | Removed | ### 3. Semantic Versioning ``` MAJOR.MINOR.PATCH MAJOR: Breaking changes (feat! or BREAKING CHANGE) MINOR: New features (feat) PATCH: Bug fixes (fix) ``` ## Implementation ### Method 1: Conventional Changelog (Node.js) ```bash # Install tools npm install -D @commitlint/cli @commitlint/config-conventional npm install -D husky npm install -D standard-version # or npm install -D semantic-release # Setup commitlint cat > commitlint.config.js << 'EOF' module.exports = { extends: ['@commitlint/config-conventional'], rules: { 'type-enum': [ 2, 'always', [ 'feat', 'fix', 'docs', 'style', 'refactor', 'perf', 'test', 'chore', 'ci', 'build', 'revert', ], ], 'subject-case': [2, 'never', ['start-case', 'pascal-case', 'upper-case']], 'subject-max-length': [2, 'always', 72], }, }; EOF # Setup husky npx husky init echo "npx --no -- commitlint --edit \$1" > .husky/commit-msg ``` ### Method 2: standard-version Configuration ```javascript // .versionrc.js module.exports = { types: [ { type: "feat", section: "Features" }, { type: "fix", section: "Bug Fixes" }, { type: "perf", section: "Performance Improvements" }, { type: "revert", section: "Reverts" }, { type: "docs", section: "Documentation", hidden: true }, { type: "style", section: "Styles", hidden: true }, { type: "chore", section: "Miscellaneous", hidden: true }, { type: "refactor", section: "Code Refactoring", hidden: true }, { type: "test", section: "Tests", hidden: true }, { type: "build", section: "Build System", hidden: true }, { type: "ci", section: "CI/CD", hidden: true }, ], commitUrlFormat: "{{host}}/{{owner}}/{{repository}}/commit/{{hash}}", compareUrlFormat: "{{host}}/{{owner}}/{{repository}}/compare/{{previousTag}}...{{currentTag}}", issueUrlFormat: "{{host}}/{{owner}}/{{repository}}/issues/{{id}}", userUrlFormat: "{{host}}/{{user}}", releaseCommitMessageFormat: "chore(release): {{currentTag}}", scripts: { prebump: 'echo "Running prebump"', postbump: 'echo "Running postbump"', prechangelog: 'echo "Running prechangelog"', postchangelog: 'echo "Running postchangelog"', }, }; ``` ```json // package.json scripts { "scripts": { "release": "standard-version", "release:minor": "standard-version --release-as minor", "release:major": "standard-version --release-as major", "release:patch": "standard-version --release-as patch", "release:dry": "standard-version --dry-run" } } ``` ### Method 3: semantic-release (Full Automation) ```javascript // release.config.js module.exports = { branches: [ "main", { name: "beta", prerelease: true }, { name: "alpha", prerelease: true }, ], plugins: [ "@semantic-release/commit-analyzer", "@semantic-release/release-notes-generator", [ "@semantic-release/changelog", { changelogFile: "CHANGELOG.md", }, ], [ "@semantic-release/npm", { npmPublish: true, }, ], [ "@semantic-release/github", { assets: ["dist/**/*.js", "dist/**/*.css"], }, ], [ "@semantic-release/git", { assets: ["CHANGELOG.md", "package.json"], message: "chore(release): ${nextRelease.version} [skip ci]\n\n${nextRelease.notes}", }, ], ], }; ``` ### Method 4: GitHub Actions Workflow ```yaml # .github/workflows/release.yml name: Release on: push: branches: [main] workflow_dispatch: inputs: release_type: description: "Release type" required: true default: "patch" type: choice options: - patch - minor - major permissions: contents: write pull-requests: write jobs: release: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 with: fetch-depth: 0 token: ${{ secrets.GITHUB_TOKEN }} - uses: actions/setup-node@v4 with: node-version: "20" cache: "npm" - run: npm ci - name: Configure Git run: | git config user.name "github-actions[bot]" git config user.email "github-actions[bot]@users.noreply.github.com" - name: Run semantic-release env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} NPM_TOKEN: ${{ secrets.NPM_TOKEN }} run: npx semantic-release # Alternative: manual release with standard-version manual-release: if: github.event_name == 'workflow_dispatch' runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 with: fetch-depth: 0 - uses: actions/setup-node@v4 with: node-version: "20" - run: npm ci - name: Configure Git run: | git config user.name "github-actions[bot]" git config user.email "github-actions[bot]@users.noreply.github.com" - name: Bump version and generate changelog run: npx standard-version --release-as ${{ inputs.release_type }} - name: Push changes run: git push --follow-tags origin main - name: Create GitHub Release uses: softprops/action-gh-release@v1 with: tag_name: ${{ steps.version.outputs.tag }} body_path: RELEASE_NOTES.md generate_release_notes: true ``` ### Method 5: git-cliff (Rust-based, Fast) ```toml # cliff.toml [changelog] header = """ # Changelog All notable changes to this project will be documented in this file. """ body = """ {% if version %}\ ## [{{ version | trim_start_matches(pat="v") }}] - {{ timestamp | date(format="%Y-%m-%d") }} {% else %}\ ## [Unreleased] {% endif %}\ {% for group, commits in commits | group_by(attribute="group") %} ### {{ group | upper_first }} {% for commit in commits %} - {% if commit.scope %}**{{ commit.scope }}:** {% endif %}\ {{ commit.message | upper_first }}\ {% if commit.github.pr_number %} ([#{{ commit.github.pr_number }}](https://github.com/owner/repo/pull/{{ commit.github.pr_number }})){% endif %}\ {% endfor %} {% endfor %} """ footer = """ {% for release in releases -%} {% if release.version -%} {% if release.previous.version -%} [{{ release.version | trim_start_matches(pat="v") }}]: \ https://github.com/owner/repo/compare/{{ release.previous.version }}...{{ release.version }} {% endif -%} {% else -%} [unreleased]: https://github.com/owner/repo/compare/{{ release.previous.version }}...HEAD {% endif -%} {% endfor %} """ trim = true [git] conventional_commits = true filter_unconventional = true split_commits = false commit_parsers = [ { message = "^feat", group = "Features" }, { message = "^fix", group = "Bug Fixes" }, { message = "^doc", group = "Documentation" }, { message = "^perf", group = "Performance" }, { message = "^refactor", group = "Refactoring" }, { message = "^style", group = "Styling" }, { message = "^test", group = "Testing" }, { message = "^chore\\(release\\)", skip = true }, { message = "^chore", group = "Miscellaneous" }, ] filter_commits = false tag_pattern = "v[0-9]*" skip_tags = "" ignore_tags = "" topo_order = false sort_commits = "oldest" [github] owner = "owner" repo = "repo" ``` ```bash # Generate changelog git cliff -o CHANGELOG.md # Generate for specific range git cliff v1.0.0..v2.0.0 -o RELEASE_NOTES.md # Preview without writing git cliff --unreleased --dry-run ``` ### Method 6: Python (commitizen) ```toml # pyproject.toml [tool.commitizen] name = "cz_conventional_commits" version = "1.0.0" version_files = [ "pyproject.toml:version", "src/__init__.py:__version__", ] tag_format = "v$version" update_changelog_on_bump = true changelog_incremental = true changelog_start_rev = "v0.1.0" [tool.commitizen.customize] message_template = "{{change_type}}{% if scope %}({{scope}}){% endif %}: {{message}}" schema = "<type>(<scope>): <subject>" schema_pattern = "^(feat|fix|docs|style|refactor|perf|test|chore)(\\(\\w+\\))?:\\s.*" bump_pattern = "^(feat|fix|perf|refactor)" bump_map = {"feat" = "MINOR", "fix" = "PATCH", "perf" = "PATCH", "refactor" = "PATCH"} ``` ```bash # Install pip install commitizen # Create commit interactively cz commit # Bump version and update changelog cz bump --changelog # Check commits cz check --rev-range HEAD~5..HEAD ``` ## Release Notes Templates ### GitHub Release Template ```markdown ## What's Changed ### 🚀 Features {{ range .Features }} - {{ .Title }} by @{{ .Author }} in #{{ .PR }} {{ end }} ### 🐛 Bug Fixes {{ range .Fixes }} - {{ .Title }} by @{{ .Author }} in #{{ .PR }} {{ end }} ### 📚 Documentation {{ range .Docs }} - {{ .Title }} by @{{ .Author }} in #{{ .PR }} {{ end }} ### 🔧 Maintenance {{ range .Chores }} - {{ .Title }} by @{{ .Author }} in #{{ .PR }} {{ end }} ## New Contributors {{ range .NewContributors }} - @{{ .Username }} made their first contribution in #{{ .PR }} {{ end }} **Full Changelog**: https://github.com/owner/repo/compare/v{{ .Previous }}...v{{ .Current }} ``` ### Internal Release Notes ```markdown # Release v2.1.0 - January 15, 2024 ## Summary This release introduces dark mode support and improves checkout performance by 40%. It also includes important security updates. ## Highlights ### 🌙 Dark Mode Users can now switch to dark mode from settings. The preference is automatically saved and synced across devices. ### ⚡ Performance - Checkout flow is 40% faster - Reduced bundle size by 15% ## Breaking Changes None in this release. ## Upgrade Guide No special steps required. Standard deployment process applies. ## Known Issues - Dark mode may flicker on initial load (fix scheduled for v2.1.1) ## Dependencies Updated | Package | From | To | Reason | | ------- | ------- | ------- | ------------------------ | | react | 18.2.0 | 18.3.0 | Performance improvements | | lodash | 4.17.20 | 4.17.21 | Security patch | ``` ## Commit Message Examples ```bash # Feature with scope feat(auth): add OAuth2 support for Google login # Bug fix with issue reference fix(checkout): resolve race condition in payment processing Closes #123 # Breaking change feat(api)!: change user endpoint response format BREAKING CHANGE: The user endpoint now returns `userId` instead of `id`. Migration guide: Update all API consumers to use the new field name. # Multiple paragraphs fix(database): handle connection timeouts gracefully Previously, connection timeouts would cause the entire request to fail without retry. This change implements exponential backoff with up to 3 retries before failing. The timeout threshold has been increased from 5s to 10s based on p99 latency analysis. Fixes #456 Reviewed-by: @alice ``` ## Best Practices ### Do's - **Follow Conventional Commits** - Enables automation - **Write clear messages** - Future you will thank you - **Reference issues** - Link commits to tickets - **Use scopes consistently** - Define team conventions - **Automate releases** - Reduce manual errors ### Don'ts - **Don't mix changes** - One logical change per commit - **Don't skip validation** - Use commitlint - **Don't manual edit** - Generated changelogs only - **Don't forget breaking changes** - Mark with `!` or footer - **Don't ignore CI** - Validate commits in pipeline ## Resources - [Keep a Changelog](https://keepachangelog.com/) - [Conventional Commits](https://www.conventionalcommits.org/) - [Semantic Versioning](https://semver.org/) - [semantic-release](https://semantic-release.gitbook.io/) - [git-cliff](https://git-cliff.org/)
👍0
👁️0
🤖 Auto-discovered
🤖system prompt•7 months ago

angular-migration

Migrate from AngularJS to Angular using hybrid mode, incremental

coding
⭐1
# Angular Migration Master AngularJS to Angular migration, including hybrid apps, component conversion, dependency injection changes, and routing migration. ## When to Use This Skill - Migrating AngularJS (1.x) applications to Angular (2+) - Running hybrid AngularJS/Angular applications - Converting directives to components - Modernizing dependency injection - Migrating routing systems - Updating to latest Angular versions - Implementing Angular best practices ## Migration Strategies ### 1. Big Bang (Complete Rewrite) - Rewrite entire app in Angular - Parallel development - Switch over at once - **Best for:** Small apps, green field projects ### 2. Incremental (Hybrid Approach) - Run AngularJS and Angular side-by-side - Migrate feature by feature - ngUpgrade for interop - **Best for:** Large apps, continuous delivery ### 3. Vertical Slice - Migrate one feature completely - New features in Angular, maintain old in AngularJS - Gradually replace - **Best for:** Medium apps, distinct features ## Hybrid App Setup ```typescript // main.ts - Bootstrap hybrid app import { platformBrowserDynamic } from "@angular/platform-browser-dynamic"; import { UpgradeModule } from "@angular/upgrade/static"; import { AppModule } from "./app/app.module"; platformBrowserDynamic() .bootstrapModule(AppModule) .then((platformRef) => { const upgrade = platformRef.injector.get(UpgradeModule); // Bootstrap AngularJS upgrade.bootstrap(document.body, ["myAngularJSApp"], { strictDi: true }); }); ``` ```typescript // app.module.ts import { NgModule } from "@angular/core"; import { BrowserModule } from "@angular/platform-browser"; import { UpgradeModule } from "@angular/upgrade/static"; @NgModule({ imports: [BrowserModule, UpgradeModule], }) export class AppModule { constructor(private upgrade: UpgradeModule) {} ngDoBootstrap() { // Bootstrapped manually in main.ts } } ``` ## Component Migration ### AngularJS Controller → Angular Component ```javascript // Before: AngularJS controller angular .module("myApp") .controller("UserController", function ($scope, UserService) { $scope.user = {}; $scope.loadUser = function (id) { UserService.getUser(id).then(function (user) { $scope.user = user; }); }; $scope.saveUser = function () { UserService.saveUser($scope.user); }; }); ``` ```typescript // After: Angular component import { Component, OnInit } from "@angular/core"; import { UserService } from "./user.service"; @Component({ selector: "app-user", template: ` <div> <h2>{{ user.name }}</h2> <button (click)="saveUser()">Save</button> </div> `, }) export class UserComponent implements OnInit { user: any = {}; constructor(private userService: UserService) {} ngOnInit() { this.loadUser(1); } loadUser(id: number) { this.userService.getUser(id).subscribe((user) => { this.user = user; }); } saveUser() { this.userService.saveUser(this.user); } } ``` ### AngularJS Directive → Angular Component ```javascript // Before: AngularJS directive angular.module("myApp").directive("userCard", function () { return { restrict: "E", scope: { user: "=", onDelete: "&", }, template: ` <div class="card"> <h3>{{ user.name }}</h3> <button ng-click="onDelete()">Delete</button> </div> `, }; }); ``` ```typescript // After: Angular component import { Component, Input, Output, EventEmitter } from "@angular/core"; @Component({ selector: "app-user-card", template: ` <div class="card"> <h3>{{ user.name }}</h3> <button (click)="delete.emit()">Delete</button> </div> `, }) export class UserCardComponent { @Input() user: any; @Output() delete = new EventEmitter<void>(); } // Usage: <app-user-card [user]="user" (delete)="handleDelete()"></app-user-card> ``` ## Service Migration ```javascript // Before: AngularJS service angular.module("myApp").factory("UserService", function ($http) { return { getUser: function (id) { return $http.get("/api/users/" + id); }, saveUser: function (user) { return $http.post("/api/users", user); }, }; }); ``` ```typescript // After: Angular service import { Injectable } from "@angular/core"; import { HttpClient } from "@angular/common/http"; import { Observable } from "rxjs"; @Injectable({ providedIn: "root", }) export class UserService { constructor(private http: HttpClient) {} getUser(id: number): Observable<any> { return this.http.get(`/api/users/${id}`); } saveUser(user: any): Observable<any> { return this.http.post("/api/users", user); } } ``` ## Dependency Injection Changes ### Downgrading Angular → AngularJS ```typescript // Angular service import { Injectable } from "@angular/core"; @Injectable({ providedIn: "root" }) export class NewService { getData() { return "data from Angular"; } } // Make available to AngularJS import { downgradeInjectable } from "@angular/upgrade/static"; angular.module("myApp").factory("newService", downgradeInjectable(NewService)); // Use in AngularJS angular.module("myApp").controller("OldController", function (newService) { console.log(newService.getData()); }); ``` ### Upgrading AngularJS → Angular ```typescript // AngularJS service angular.module('myApp').factory('oldService', function() { return { getData: function() { return 'data from AngularJS'; } }; }); // Make available to Angular import { InjectionToken } from '@angular/core'; export const OLD_SERVICE = new InjectionToken<any>('oldService'); @NgModule({ providers: [ { provide: OLD_SERVICE, useFactory: (i: any) => i.get('oldService'), deps: ['$injector'] } ] }) // Use in Angular @Component({...}) export class NewComponent { constructor(@Inject(OLD_SERVICE) private oldService: any) { console.log(this.oldService.getData()); } } ``` ## Routing Migration ```javascript // Before: AngularJS routing angular.module("myApp").config(function ($routeProvider) { $routeProvider .when("/users", { template: "<user-list></user-list>", }) .when("/users/:id", { template: "<user-detail></user-detail>", }); }); ``` ```typescript // After: Angular routing import { NgModule } from "@angular/core"; import { RouterModule, Routes } from "@angular/router"; const routes: Routes = [ { path: "users", component: UserListComponent }, { path: "users/:id", component: UserDetailComponent }, ]; @NgModule({ imports: [RouterModule.forRoot(routes)], exports: [RouterModule], }) export class AppRoutingModule {} ``` ## Forms Migration ```html <!-- Before: AngularJS --> <form name="userForm" ng-submit="saveUser()"> <input type="text" ng-model="user.name" required /> <input type="email" ng-model="user.email" required /> <button ng-disabled="userForm.$invalid">Save</button> </form> ``` ```typescript // After: Angular (Template-driven) @Component({ template: ` <form #userForm="ngForm" (ngSubmit)="saveUser()"> <input type="text" [(ngModel)]="user.name" name="name" required> <input type="email" [(ngModel)]="user.email" name="email" required> <button [disabled]="userForm.invalid">Save</button> </form> ` }) // Or Reactive Forms (preferred) import { FormBuilder, FormGroup, Validators } from '@angular/forms'; @Component({ template: ` <form [formGroup]="userForm" (ngSubmit)="saveUser()"> <input formControlName="name"> <input formControlName="email"> <button [disabled]="userForm.invalid">Save</button> </form> ` }) export class UserFormComponent { userForm: FormGroup; constructor(private fb: FormBuilder) { this.userForm = this.fb.group({ name: ['', Validators.required], email: ['', [Validators.required, Validators.email]] }); } saveUser() { console.log(this.userForm.value); } } ``` ## Migration Timeline ``` Phase 1: Setup (1-2 weeks) - Install Angular CLI - Set up hybrid app - Configure build tools - Set up testing Phase 2: Infrastructure (2-4 weeks) - Migrate services - Migrate utilities - Set up routing - Migrate shared components Phase 3: Feature Migration (varies) - Migrate feature by feature - Test thoroughly - Deploy incrementally Phase 4: Cleanup (1-2 weeks) - Remove AngularJS code - Remove ngUpgrade - Optimize bundle - Final testing ``` ## Resources - **references/hybrid-mode.md**: Hybrid app patterns - **references/component-migration.md**: Component conversion guide - **references/dependency-injection.md**: DI migration strategies - **references/routing.md**: Routing migration - **assets/hybrid-bootstrap.ts**: Hybrid app template - **assets/migration-timeline.md**: Project planning - **scripts/analyze-angular-app.sh**: App analysis script ## Best Practices 1. **Start with Services**: Migrate services first (easier) 2. **Incremental Approach**: Feature-by-feature migration 3. **Test Continuously**: Test at every step 4. **Use TypeScript**: Migrate to TypeScript early 5. **Follow Style Guide**: Angular style guide from day 1 6. **Optimize Later**: Get it working, then optimize 7. **Document**: Keep migration notes ## Common Pitfalls - Not setting up hybrid app correctly - Migrating UI before logic - Ignoring change detection differences - Not handling scope properly - Mixing patterns (AngularJS + Angular) - Inadequate testing
👍0
👁️0
🤖 Auto-discovered
🤖system prompt•7 months ago

database-migration

Execute database migrations across ORMs and platforms with

data
⭐1
# Database Migration Master database schema and data migrations across ORMs (Sequelize, TypeORM, Prisma), including rollback strategies and zero-downtime deployments. ## When to Use This Skill - Migrating between different ORMs - Performing schema transformations - Moving data between databases - Implementing rollback procedures - Zero-downtime deployments - Database version upgrades - Data model refactoring ## ORM Migrations ### Sequelize Migrations ```javascript // migrations/20231201-create-users.js module.exports = { up: async (queryInterface, Sequelize) => { await queryInterface.createTable("users", { id: { type: Sequelize.INTEGER, primaryKey: true, autoIncrement: true, }, email: { type: Sequelize.STRING, unique: true, allowNull: false, }, createdAt: Sequelize.DATE, updatedAt: Sequelize.DATE, }); }, down: async (queryInterface, Sequelize) => { await queryInterface.dropTable("users"); }, }; // Run: npx sequelize-cli db:migrate // Rollback: npx sequelize-cli db:migrate:undo ``` ### TypeORM Migrations ```typescript // migrations/1701234567-CreateUsers.ts import { MigrationInterface, QueryRunner, Table } from "typeorm"; export class CreateUsers1701234567 implements MigrationInterface { public async up(queryRunner: QueryRunner): Promise<void> { await queryRunner.createTable( new Table({ name: "users", columns: [ { name: "id", type: "int", isPrimary: true, isGenerated: true, generationStrategy: "increment", }, { name: "email", type: "varchar", isUnique: true, }, { name: "created_at", type: "timestamp", default: "CURRENT_TIMESTAMP", }, ], }), ); } public async down(queryRunner: QueryRunner): Promise<void> { await queryRunner.dropTable("users"); } } // Run: npm run typeorm migration:run // Rollback: npm run typeorm migration:revert ``` ### Prisma Migrations ```prisma // schema.prisma model User { id Int @id @default(autoincrement()) email String @unique createdAt DateTime @default(now()) } // Generate migration: npx prisma migrate dev --name create_users // Apply: npx prisma migrate deploy ``` ## Schema Transformations ### Adding Columns with Defaults ```javascript // Safe migration: add column with default module.exports = { up: async (queryInterface, Sequelize) => { await queryInterface.addColumn("users", "status", { type: Sequelize.STRING, defaultValue: "active", allowNull: false, }); }, down: async (queryInterface) => { await queryInterface.removeColumn("users", "status"); }, }; ``` ### Renaming Columns (Zero Downtime) ```javascript // Step 1: Add new column module.exports = { up: async (queryInterface, Sequelize) => { await queryInterface.addColumn("users", "full_name", { type: Sequelize.STRING, }); // Copy data from old column await queryInterface.sequelize.query("UPDATE users SET full_name = name"); }, down: async (queryInterface) => { await queryInterface.removeColumn("users", "full_name"); }, }; // Step 2: Update application to use new column // Step 3: Remove old column module.exports = { up: async (queryInterface) => { await queryInterface.removeColumn("users", "name"); }, down: async (queryInterface, Sequelize) => { await queryInterface.addColumn("users", "name", { type: Sequelize.STRING, }); }, }; ``` ### Changing Column Types ```javascript module.exports = { up: async (queryInterface, Sequelize) => { // For large tables, use multi-step approach // 1. Add new column await queryInterface.addColumn("users", "age_new", { type: Sequelize.INTEGER, }); // 2. Copy and transform data await queryInterface.sequelize.query(` UPDATE users SET age_new = CAST(age AS INTEGER) WHERE age IS NOT NULL `); // 3. Drop old column await queryInterface.removeColumn("users", "age"); // 4. Rename new column await queryInterface.renameColumn("users", "age_new", "age"); }, down: async (queryInterface, Sequelize) => { await queryInterface.changeColumn("users", "age", { type: Sequelize.STRING, }); }, }; ``` ## Data Transformations ### Complex Data Migration ```javascript module.exports = { up: async (queryInterface, Sequelize) => { // Get all records const [users] = await queryInterface.sequelize.query( "SELECT id, address_string FROM users", ); // Transform each record for (const user of users) { const addressParts = user.address_string.split(","); await queryInterface.sequelize.query( `UPDATE users SET street = :street, city = :city, state = :state WHERE id = :id`, { replacements: { id: user.id, street: addressParts[0]?.trim(), city: addressParts[1]?.trim(), state: addressParts[2]?.trim(), }, }, ); } // Drop old column await queryInterface.removeColumn("users", "address_string"); }, down: async (queryInterface, Sequelize) => { // Reconstruct original column await queryInterface.addColumn("users", "address_string", { type: Sequelize.STRING, }); await queryInterface.sequelize.query(` UPDATE users SET address_string = CONCAT(street, ', ', city, ', ', state) `); await queryInterface.removeColumn("users", "street"); await queryInterface.removeColumn("users", "city"); await queryInterface.removeColumn("users", "state"); }, }; ``` ## Rollback Strategies ### Transaction-Based Migrations ```javascript module.exports = { up: async (queryInterface, Sequelize) => { const transaction = await queryInterface.sequelize.transaction(); try { await queryInterface.addColumn( "users", "verified", { type: Sequelize.BOOLEAN, defaultValue: false }, { transaction }, ); await queryInterface.sequelize.query( "UPDATE users SET verified = true WHERE email_verified_at IS NOT NULL", { transaction }, ); await transaction.commit(); } catch (error) { await transaction.rollback(); throw error; } }, down: async (queryInterface) => { await queryInterface.removeColumn("users", "verified"); }, }; ``` ### Checkpoint-Based Rollback ```javascript module.exports = { up: async (queryInterface, Sequelize) => { // Create backup table await queryInterface.sequelize.query( "CREATE TABLE users_backup AS SELECT * FROM users", ); try { // Perform migration await queryInterface.addColumn("users", "new_field", { type: Sequelize.STRING, }); // Verify migration const [result] = await queryInterface.sequelize.query( "SELECT COUNT(*) as count FROM users WHERE new_field IS NULL", ); if (result[0].count > 0) { throw new Error("Migration verification failed"); } // Drop backup await queryInterface.dropTable("users_backup"); } catch (error) { // Restore from backup await queryInterface.sequelize.query("DROP TABLE users"); await queryInterface.sequelize.query( "CREATE TABLE users AS SELECT * FROM users_backup", ); await queryInterface.dropTable("users_backup"); throw error; } }, }; ``` ## Zero-Downtime Migrations ### Blue-Green Deployment Strategy ```javascript // Phase 1: Make changes backward compatible module.exports = { up: async (queryInterface, Sequelize) => { // Add new column (both old and new code can work) await queryInterface.addColumn("users", "email_new", { type: Sequelize.STRING, }); }, }; // Phase 2: Deploy code that writes to both columns // Phase 3: Backfill data module.exports = { up: async (queryInterface) => { await queryInterface.sequelize.query(` UPDATE users SET email_new = email WHERE email_new IS NULL `); }, }; // Phase 4: Deploy code that reads from new column // Phase 5: Remove old column module.exports = { up: async (queryInterface) => { await queryInterface.removeColumn("users", "email"); }, }; ``` ## Cross-Database Migrations ### PostgreSQL to MySQL ```javascript // Handle differences module.exports = { up: async (queryInterface, Sequelize) => { const dialectName = queryInterface.sequelize.getDialect(); if (dialectName === "mysql") { await queryInterface.createTable("users", { id: { type: Sequelize.INTEGER, primaryKey: true, autoIncrement: true, }, data: { type: Sequelize.JSON, // MySQL JSON type }, }); } else if (dialectName === "postgres") { await queryInterface.createTable("users", { id: { type: Sequelize.INTEGER, primaryKey: true, autoIncrement: true, }, data: { type: Sequelize.JSONB, // PostgreSQL JSONB type }, }); } }, }; ``` ## Resources - **references/orm-switching.md**: ORM migration guides - **references/schema-migration.md**: Schema transformation patterns - **references/data-transformation.md**: Data migration scripts - **references/rollback-strategies.md**: Rollback procedures - **assets/schema-migration-template.sql**: SQL migration templates - **assets/data-migration-script.py**: Data migration utilities - **scripts/test-migration.sh**: Migration testing script ## Best Practices 1. **Always Provide Rollback**: Every up() needs a down() 2. **Test Migrations**: Test on staging first 3. **Use Transactions**: Atomic migrations when possible 4. **Backup First**: Always backup before migration 5. **Small Changes**: Break into small, incremental steps 6. **Monitor**: Watch for errors during deployment 7. **Document**: Explain why and how 8. **Idempotent**: Migrations should be rerunnable ## Common Pitfalls - Not testing rollback procedures - Making breaking changes without downtime strategy - Forgetting to handle NULL values - Not considering index performance - Ignoring foreign key constraints - Migrating too much data at once
👍0
👁️0
🤖 Auto-discovered
🤖system prompt•7 months ago

dependency-upgrade

Manage major dependency version upgrades with compatibility

coding
⭐1
# Dependency Upgrade Master major dependency version upgrades, compatibility analysis, staged upgrade strategies, and comprehensive testing approaches. ## When to Use This Skill - Upgrading major framework versions - Updating security-vulnerable dependencies - Modernizing legacy dependencies - Resolving dependency conflicts - Planning incremental upgrade paths - Testing compatibility matrices - Automating dependency updates ## Semantic Versioning Review ``` MAJOR.MINOR.PATCH (e.g., 2.3.1) MAJOR: Breaking changes MINOR: New features, backward compatible PATCH: Bug fixes, backward compatible ^2.3.1 = >=2.3.1 <3.0.0 (minor updates) ~2.3.1 = >=2.3.1 <2.4.0 (patch updates) 2.3.1 = exact version ``` ## Dependency Analysis ### Audit Dependencies ```bash # npm npm outdated npm audit npm audit fix # yarn yarn outdated yarn audit # Check for major updates npx npm-check-updates npx npm-check-updates -u # Update package.json ``` ### Analyze Dependency Tree ```bash # See why a package is installed npm ls package-name yarn why package-name # Find duplicate packages npm dedupe yarn dedupe # Visualize dependencies npx madge --image graph.png src/ ``` ## Compatibility Matrix ```javascript // compatibility-matrix.js const compatibilityMatrix = { react: { "16.x": { "react-dom": "^16.0.0", "react-router-dom": "^5.0.0", "@testing-library/react": "^11.0.0", }, "17.x": { "react-dom": "^17.0.0", "react-router-dom": "^5.0.0 || ^6.0.0", "@testing-library/react": "^12.0.0", }, "18.x": { "react-dom": "^18.0.0", "react-router-dom": "^6.0.0", "@testing-library/react": "^13.0.0", }, }, }; function checkCompatibility(packages) { // Validate package versions against matrix } ``` ## Staged Upgrade Strategy ### Phase 1: Planning ```bash # 1. Identify current versions npm list --depth=0 # 2. Check for breaking changes # Read CHANGELOG.md and MIGRATION.md # 3. Create upgrade plan echo "Upgrade order: 1. TypeScript 2. React 3. React Router 4. Testing libraries 5. Build tools" > UPGRADE_PLAN.md ``` ### Phase 2: Incremental Updates ```bash # Don't upgrade everything at once! # Step 1: Update TypeScript npm install typescript@latest # Test npm run test npm run build # Step 2: Update React (one major version at a time) npm install react@17 react-dom@17 # Test again npm run test # Step 3: Continue with other packages npm install react-router-dom@6 # And so on... ``` ### Phase 3: Validation ```javascript // tests/compatibility.test.js describe("Dependency Compatibility", () => { it("should have compatible React versions", () => { const reactVersion = require("react/package.json").version; const reactDomVersion = require("react-dom/package.json").version; expect(reactVersion).toBe(reactDomVersion); }); it("should not have peer dependency warnings", () => { // Run npm ls and check for warnings }); }); ``` ## Breaking Change Handling ### Identifying Breaking Changes ```bash # Check the changelog directly curl https://raw.githubusercontent.com/facebook/react/master/CHANGELOG.md ``` ### Codemod for Automated Fixes ```bash # Run jscodeshift with transform URL npx jscodeshift -t <transform-url> <path> # Example: Rename unsafe lifecycle methods npx jscodeshift -t https://raw.githubusercontent.com/reactjs/react-codemod/master/transforms/rename-unsafe-lifecycles.js src/ # For TypeScript files npx jscodeshift -t https://raw.githubusercontent.com/reactjs/react-codemod/master/transforms/rename-unsafe-lifecycles.js --parser=tsx src/ # Dry run to preview changes npx jscodeshift -t https://raw.githubusercontent.com/reactjs/react-codemod/master/transforms/rename-unsafe-lifecycles.js --dry src/ ``` ### Custom Migration Script ```javascript // migration-script.js const fs = require("fs"); const glob = require("glob"); glob("src/**/*.tsx", (err, files) => { files.forEach((file) => { let content = fs.readFileSync(file, "utf8"); // Replace old API with new API content = content.replace( /componentWillMount/g, "UNSAFE_componentWillMount", ); // Update imports content = content.replace( /import { Component } from 'react'/g, "import React, { Component } from 'react'", ); fs.writeFileSync(file, content); }); }); ``` ## Testing Strategy ### Unit Tests ```javascript // Ensure tests pass before and after upgrade npm run test // Update test utilities if needed npm install @testing-library/react@latest ``` ### Integration Tests ```javascript // tests/integration/app.test.js describe("App Integration", () => { it("should render without crashing", () => { render(<App />); }); it("should handle navigation", () => { const { getByText } = render(<App />); fireEvent.click(getByText("Navigate")); expect(screen.getByText("New Page")).toBeInTheDocument(); }); }); ``` ### Visual Regression Tests ```javascript // visual-regression.test.js describe("Visual Regression", () => { it("should match snapshot", () => { const { container } = render(<App />); expect(container.firstChild).toMatchSnapshot(); }); }); ``` ### E2E Tests ```javascript // cypress/e2e/app.cy.js describe("E2E Tests", () => { it("should complete user flow", () => { cy.visit("/"); cy.get('[data-testid="login"]').click(); cy.get('input[name="email"]').type("user@example.com"); cy.get('button[type="submit"]').click(); cy.url().should("include", "/dashboard"); }); }); ``` ## Automated Dependency Updates ### Renovate Configuration ```json // renovate.json { "extends": ["config:base"], "packageRules": [ { "matchUpdateTypes": ["minor", "patch"], "automerge": true }, { "matchUpdateTypes": ["major"], "automerge": false, "labels": ["major-update"] } ], "schedule": ["before 3am on Monday"], "timezone": "America/New_York" } ``` ### Dependabot Configuration ```yaml # .github/dependabot.yml version: 2 updates: - package-ecosystem: "npm" directory: "/" schedule: interval: "weekly" open-pull-requests-limit: 5 reviewers: - "team-leads" commit-message: prefix: "chore" include: "scope" ``` ## Rollback Plan ```javascript // rollback.sh #!/bin/bash # Save current state git stash git checkout -b upgrade-branch # Attempt upgrade npm install package@latest # Run tests if npm run test; then echo "Upgrade successful" git add package.json package-lock.json git commit -m "chore: upgrade package" else echo "Upgrade failed, rolling back" git checkout main git branch -D upgrade-branch npm install # Restore from package-lock.json fi ``` ## Common Upgrade Patterns ### Lock File Management ```bash # npm npm install --package-lock-only # Update lock file only npm ci # Clean install from lock file # yarn yarn install --frozen-lockfile # CI mode yarn upgrade-interactive # Interactive upgrades ``` ### Peer Dependency Resolution ```bash # npm 7+: strict peer dependencies npm install --legacy-peer-deps # Ignore peer deps # npm 8+: override peer dependencies npm install --force ``` ### Workspace Upgrades ```bash # Update all workspace packages npm install --workspaces # Update specific workspace npm install package@latest --workspace=packages/app ``` ## Resources - **references/semver.md**: Semantic versioning guide - **references/compatibility-matrix.md**: Common compatibility issues - **references/staged-upgrades.md**: Incremental upgrade strategies - **references/testing-strategy.md**: Comprehensive testing approaches - **assets/upgrade-checklist.md**: Step-by-step checklist - **assets/compatibility-matrix.csv**: Version compatibility table - **scripts/audit-dependencies.sh**: Dependency audit script ## Best Practices 1. **Read Changelogs**: Understand what changed 2. **Upgrade Incrementally**: One major version at a time 3. **Test Thoroughly**: Unit, integration, E2E tests 4. **Check Peer Dependencies**: Resolve conflicts early 5. **Use Lock Files**: Ensure reproducible installs 6. **Automate Updates**: Use Renovate or Dependabot 7. **Monitor**: Watch for runtime errors post-upgrade 8. **Document**: Keep upgrade notes ## Upgrade Checklist ```markdown Pre-Upgrade: - [ ] Review current dependency versions - [ ] Read changelogs for breaking changes - [ ] Create feature branch - [ ] Backup current state (git tag) - [ ] Run full test suite (baseline) During Upgrade: - [ ] Upgrade one dependency at a time - [ ] Update peer dependencies - [ ] Fix TypeScript errors - [ ] Update tests if needed - [ ] Run test suite after each upgrade - [ ] Check bundle size impact Post-Upgrade: - [ ] Full regression testing - [ ] Performance testing - [ ] Update documentation - [ ] Deploy to staging - [ ] Monitor for errors - [ ] Deploy to production ``` ## Common Pitfalls - Upgrading all dependencies at once - Not testing after each upgrade - Ignoring peer dependency warnings - Forgetting to update lock file - Not reading breaking change notes - Skipping major versions - Not having rollback plan
👍0
👁️0
🤖 Auto-discovered
🤖system prompt•7 months ago

react-modernization

Upgrade React applications to latest versions, migrate from class

coding
⭐1
# React Modernization Master React version upgrades, class to hooks migration, concurrent features adoption, and codemods for automated transformation. ## When to Use This Skill - Upgrading React applications to latest versions - Migrating class components to functional components with hooks - Adopting concurrent React features (Suspense, transitions) - Applying codemods for automated refactoring - Modernizing state management patterns - Updating to TypeScript - Improving performance with React 18+ features ## Version Upgrade Path ### React 16 → 17 → 18 **Breaking Changes by Version:** **React 17:** - Event delegation changes - No event pooling - Effect cleanup timing - JSX transform (no React import needed) **React 18:** - Automatic batching - Concurrent rendering - Strict Mode changes (double invocation) - New root API - Suspense on server ## Class to Hooks Migration ### State Management ```javascript // Before: Class component class Counter extends React.Component { constructor(props) { super(props); this.state = { count: 0, name: "", }; } increment = () => { this.setState({ count: this.state.count + 1 }); }; render() { return ( <div> <p>Count: {this.state.count}</p> <button onClick={this.increment}>Increment</button> </div> ); } } // After: Functional component with hooks function Counter() { const [count, setCount] = useState(0); const [name, setName] = useState(""); const increment = () => { setCount(count + 1); }; return ( <div> <p>Count: {count}</p> <button onClick={increment}>Increment</button> </div> ); } ``` ### Lifecycle Methods to Hooks ```javascript // Before: Lifecycle methods class DataFetcher extends React.Component { state = { data: null, loading: true }; componentDidMount() { this.fetchData(); } componentDidUpdate(prevProps) { if (prevProps.id !== this.props.id) { this.fetchData(); } } componentWillUnmount() { this.cancelRequest(); } fetchData = async () => { const data = await fetch(`/api/${this.props.id}`); this.setState({ data, loading: false }); }; cancelRequest = () => { // Cleanup }; render() { if (this.state.loading) return <div>Loading...</div>; return <div>{this.state.data}</div>; } } // After: useEffect hook function DataFetcher({ id }) { const [data, setData] = useState(null); const [loading, setLoading] = useState(true); useEffect(() => { let cancelled = false; const fetchData = async () => { try { const response = await fetch(`/api/${id}`); const result = await response.json(); if (!cancelled) { setData(result); setLoading(false); } } catch (error) { if (!cancelled) { console.error(error); } } }; fetchData(); // Cleanup function return () => { cancelled = true; }; }, [id]); // Re-run when id changes if (loading) return <div>Loading...</div>; return <div>{data}</div>; } ``` ### Context and HOCs to Hooks ```javascript // Before: Context consumer and HOC const ThemeContext = React.createContext(); class ThemedButton extends React.Component { static contextType = ThemeContext; render() { return ( <button style={{ background: this.context.theme }}> {this.props.children} </button> ); } } // After: useContext hook function ThemedButton({ children }) { const { theme } = useContext(ThemeContext); return <button style={{ background: theme }}>{children}</button>; } // Before: HOC for data fetching function withUser(Component) { return class extends React.Component { state = { user: null }; componentDidMount() { fetchUser().then((user) => this.setState({ user })); } render() { return <Component {...this.props} user={this.state.user} />; } }; } // After: Custom hook function useUser() { const [user, setUser] = useState(null); useEffect(() => { fetchUser().then(setUser); }, []); return user; } function UserProfile() { const user = useUser(); if (!user) return <div>Loading...</div>; return <div>{user.name}</div>; } ``` ## React 18 Concurrent Features ### New Root API ```javascript // Before: React 17 import ReactDOM from "react-dom"; ReactDOM.render(<App />, document.getElementById("root")); // After: React 18 import { createRoot } from "react-dom/client"; const root = createRoot(document.getElementById("root")); root.render(<App />); ``` ### Automatic Batching ```javascript // React 18: All updates are batched function handleClick() { setCount((c) => c + 1); setFlag((f) => !f); // Only one re-render (batched) } // Even in async: setTimeout(() => { setCount((c) => c + 1); setFlag((f) => !f); // Still batched in React 18! }, 1000); // Opt out if needed import { flushSync } from "react-dom"; flushSync(() => { setCount((c) => c + 1); }); // Re-render happens here setFlag((f) => !f); // Another re-render ``` ### Transitions ```javascript import { useState, useTransition } from "react"; function SearchResults() { const [query, setQuery] = useState(""); const [results, setResults] = useState([]); const [isPending, startTransition] = useTransition(); const handleChange = (e) => { // Urgent: Update input immediately setQuery(e.target.value); // Non-urgent: Update results (can be interrupted) startTransition(() => { setResults(searchResults(e.target.value)); }); }; return ( <> <input value={query} onChange={handleChange} /> {isPending && <Spinner />} <Results data={results} /> </> ); } ``` ### Suspense for Data Fetching ```javascript import { Suspense } from "react"; // Resource-based data fetching (with React 18) const resource = fetchProfileData(); function ProfilePage() { return ( <Suspense fallback={<Loading />}> <ProfileDetails /> <Suspense fallback={<Loading />}> <ProfileTimeline /> </Suspense> </Suspense> ); } function ProfileDetails() { // This will suspend if data not ready const user = resource.user.read(); return <h1>{user.name}</h1>; } function ProfileTimeline() { const posts = resource.posts.read(); return <Timeline posts={posts} />; } ``` ## Codemods for Automation ### Run React Codemods ```bash # Rename unsafe lifecycle methods npx jscodeshift -t https://raw.githubusercontent.com/reactjs/react-codemod/master/transforms/rename-unsafe-lifecycles.js src/ # Update React imports (React 17+) npx jscodeshift -t https://raw.githubusercontent.com/reactjs/react-codemod/master/transforms/update-react-imports.js src/ # Add error boundaries npx jscodeshift -t https://raw.githubusercontent.com/reactjs/react-codemod/master/transforms/error-boundaries.js src/ # For TypeScript files npx jscodeshift -t https://raw.githubusercontent.com/reactjs/react-codemod/master/transforms/rename-unsafe-lifecycles.js --parser=tsx src/ # Dry run to preview changes npx jscodeshift -t https://raw.githubusercontent.com/reactjs/react-codemod/master/transforms/rename-unsafe-lifecycles.js --dry --print src/ # Class to Hooks (third-party) npx codemod react/hooks/convert-class-to-function src/ ``` ### Custom Codemod Example ```javascript // custom-codemod.js module.exports = function (file, api) { const j = api.jscodeshift; const root = j(file.source); // Find setState calls root .find(j.CallExpression, { callee: { type: "MemberExpression", property: { name: "setState" }, }, }) .forEach((path) => { // Transform to useState // ... transformation logic }); return root.toSource(); }; // Run: jscodeshift -t custom-codemod.js src/ ``` ## Performance Optimization ### useMemo and useCallback ```javascript function ExpensiveComponent({ items, filter }) { // Memoize expensive calculation const filteredItems = useMemo(() => { return items.filter((item) => item.category === filter); }, [items, filter]); // Memoize callback to prevent child re-renders const handleClick = useCallback((id) => { console.log("Clicked:", id); }, []); // No dependencies, never changes return <List items={filteredItems} onClick={handleClick} />; } // Child component with memo const List = React.memo(({ items, onClick }) => { return items.map((item) => ( <Item key={item.id} item={item} onClick={onClick} /> )); }); ``` ### Code Splitting ```javascript import { lazy, Suspense } from "react"; // Lazy load components const Dashboard = lazy(() => import("./Dashboard")); const Settings = lazy(() => import("./Settings")); function App() { return ( <Suspense fallback={<Loading />}> <Routes> <Route path="/dashboard" element={<Dashboard />} /> <Route path="/settings" element={<Settings />} /> </Routes> </Suspense> ); } ``` ## TypeScript Migration ```typescript // Before: JavaScript function Button({ onClick, children }) { return <button onClick={onClick}>{children}</button>; } // After: TypeScript interface ButtonProps { onClick: () => void; children: React.ReactNode; } function Button({ onClick, children }: ButtonProps) { return <button onClick={onClick}>{children}</button>; } // Generic components interface ListProps<T> { items: T[]; renderItem: (item: T) => React.ReactNode; } function List<T>({ items, renderItem }: ListProps<T>) { return <>{items.map(renderItem)}</>; } ``` ## Migration Checklist ```markdown ### Pre-Migration - [ ] Update dependencies incrementally (not all at once) - [ ] Review breaking changes in release notes - [ ] Set up testing suite - [ ] Create feature branch ### Class → Hooks Migration - [ ] Identify class components to migrate - [ ] Start with leaf components (no children) - [ ] Convert state to useState - [ ] Convert lifecycle to useEffect - [ ] Convert context to useContext - [ ] Extract custom hooks - [ ] Test thoroughly ### React 18 Upgrade - [ ] Update to React 17 first (if needed) - [ ] Update react and react-dom to 18 - [ ] Update @types/react if using TypeScript - [ ] Change to createRoot API - [ ] Test with StrictMode (double invocation) - [ ] Address concurrent rendering issues - [ ] Adopt Suspense/Transitions where beneficial ### Performance - [ ] Identify performance bottlenecks - [ ] Add React.memo where appropriate - [ ] Use useMemo/useCallback for expensive operations - [ ] Implement code splitting - [ ] Optimize re-renders ### Testing - [ ] Update test utilities (React Testing Library) - [ ] Test with React 18 features - [ ] Check for warnings in console - [ ] Performance testing ``` ## Resources - **references/breaking-changes.md**: Version-specific breaking changes - **references/codemods.md**: Codemod usage guide - **references/hooks-migration.md**: Comprehensive hooks patterns - **references/concurrent-features.md**: React 18 concurrent features - **assets/codemod-config.json**: Codemod configurations - **assets/migration-checklist.md**: Step-by-step checklist - **scripts/apply-codemods.sh**: Automated codemod script ## Best Practices 1. **Incremental Migration**: Don't migrate everything at once 2. **Test Thoroughly**: Comprehensive testing at each step 3. **Use Codemods**: Automate repetitive transformations 4. **Start Simple**: Begin with leaf components 5. **Leverage StrictMode**: Catch issues early 6. **Monitor Performance**: Measure before and after 7. **Document Changes**: Keep migration log ## Common Pitfalls - Forgetting useEffect dependencies - Over-using useMemo/useCallback - Not handling cleanup in useEffect - Mixing class and functional patterns - Ignoring StrictMode warnings - Breaking change assumptions
👍0
👁️0
🤖 Auto-discovered
🤖system prompt•7 months ago

gdpr-data-handling

Implement GDPR-compliant data handling with consent management,

data
⭐1
# GDPR Data Handling Practical implementation guide for GDPR-compliant data processing, consent management, and privacy controls. ## When to Use This Skill - Building systems that process EU personal data - Implementing consent management - Handling data subject requests (DSRs) - Conducting GDPR compliance reviews - Designing privacy-first architectures - Creating data processing agreements ## Core Concepts ### 1. Personal Data Categories | Category | Examples | Protection Level | | ---------------------- | --------------------------- | ------------------ | | **Basic** | Name, email, phone | Standard | | **Sensitive (Art. 9)** | Health, religion, ethnicity | Explicit consent | | **Criminal (Art. 10)** | Convictions, offenses | Official authority | | **Children's** | Under 16 data | Parental consent | ### 2. Legal Bases for Processing ``` Article 6 - Lawful Bases: ├── Consent: Freely given, specific, informed ├── Contract: Necessary for contract performance ├── Legal Obligation: Required by law ├── Vital Interests: Protecting someone's life ├── Public Interest: Official functions └── Legitimate Interest: Balanced against rights ``` ### 3. Data Subject Rights ``` Right to Access (Art. 15) ─┐ Right to Rectification (Art. 16) │ Right to Erasure (Art. 17) │ Must respond Right to Restrict (Art. 18) │ within 1 month Right to Portability (Art. 20) │ Right to Object (Art. 21) ─┘ ``` ## Implementation Patterns ### Pattern 1: Consent Management ```javascript // Consent data model const consentSchema = { userId: String, consents: [ { purpose: String, // 'marketing', 'analytics', etc. granted: Boolean, timestamp: Date, source: String, // 'web_form', 'api', etc. version: String, // Privacy policy version ipAddress: String, // For proof userAgent: String, // For proof }, ], auditLog: [ { action: String, // 'granted', 'withdrawn', 'updated' purpose: String, timestamp: Date, source: String, }, ], }; // Consent service class ConsentManager { async recordConsent(userId, purpose, granted, metadata) { const consent = { purpose, granted, timestamp: new Date(), source: metadata.source, version: await this.getCurrentPolicyVersion(), ipAddress: metadata.ipAddress, userAgent: metadata.userAgent, }; // Store consent await this.db.consents.updateOne( { userId }, { $push: { consents: consent, auditLog: { action: granted ? "granted" : "withdrawn", purpose, timestamp: consent.timestamp, source: metadata.source, }, }, }, { upsert: true }, ); // Emit event for downstream systems await this.eventBus.emit("consent.changed", { userId, purpose, granted, timestamp: consent.timestamp, }); } async hasConsent(userId, purpose) { const record = await this.db.consents.findOne({ userId }); if (!record) return false; const latestConsent = record.consents .filter((c) => c.purpose === purpose) .sort((a, b) => b.timestamp - a.timestamp)[0]; return latestConsent?.granted === true; } async getConsentHistory(userId) { const record = await this.db.consents.findOne({ userId }); return record?.auditLog || []; } } ``` ```html <!-- GDPR-compliant consent UI --> <div class="consent-banner" role="dialog" aria-labelledby="consent-title"> <h2 id="consent-title">Cookie Preferences</h2> <p> We use cookies to improve your experience. Select your preferences below. </p> <form id="consent-form"> <!-- Necessary - always on, no consent needed --> <div class="consent-category"> <input type="checkbox" id="necessary" checked disabled /> <label for="necessary"> <strong>Necessary</strong> <span>Required for the website to function. Cannot be disabled.</span> </label> </div> <!-- Analytics - requires consent --> <div class="consent-category"> <input type="checkbox" id="analytics" name="analytics" /> <label for="analytics"> <strong>Analytics</strong> <span>Help us understand how you use our site.</span> </label> </div> <!-- Marketing - requires consent --> <div class="consent-category"> <input type="checkbox" id="marketing" name="marketing" /> <label for="marketing"> <strong>Marketing</strong> <span>Personalized ads based on your interests.</span> </label> </div> <div class="consent-actions"> <button type="button" id="accept-all">Accept All</button> <button type="button" id="reject-all">Reject All</button> <button type="submit">Save Preferences</button> </div> <p class="consent-links"> <a href="/privacy-policy">Privacy Policy</a> | <a href="/cookie-policy">Cookie Policy</a> </p> </form> </div> ``` ### Pattern 2: Data Subject Access Request (DSAR) ```python from datetime import datetime, timedelta from typing import Dict, List, Optional import json class DSARHandler: """Handle Data Subject Access Requests.""" RESPONSE_DEADLINE_DAYS = 30 EXTENSION_ALLOWED_DAYS = 60 # For complex requests def __init__(self, data_sources: List['DataSource']): self.data_sources = data_sources async def submit_request( self, request_type: str, # 'access', 'erasure', 'rectification', 'portability' user_id: str, verified: bool, details: Optional[Dict] = None ) -> str: """Submit a new DSAR.""" request = { 'id': self.generate_request_id(), 'type': request_type, 'user_id': user_id, 'status': 'pending_verification' if not verified else 'processing', 'submitted_at': datetime.utcnow(), 'deadline': datetime.utcnow() + timedelta(days=self.RESPONSE_DEADLINE_DAYS), 'details': details or {}, 'audit_log': [{ 'action': 'submitted', 'timestamp': datetime.utcnow(), 'details': 'Request received' }] } await self.db.dsar_requests.insert_one(request) await self.notify_dpo(request) return request['id'] async def process_access_request(self, request_id: str) -> Dict: """Process a data access request.""" request = await self.get_request(request_id) if request['type'] != 'access': raise ValueError("Not an access request") # Collect data from all sources user_data = {} for source in self.data_sources: try: data = await source.get_user_data(request['user_id']) user_data[source.name] = data except Exception as e: user_data[source.name] = {'error': str(e)} # Format response response = { 'request_id': request_id, 'generated_at': datetime.utcnow().isoformat(), 'data_categories': list(user_data.keys()), 'data': user_data, 'retention_info': await self.get_retention_info(), 'processing_purposes': await self.get_processing_purposes(), 'third_party_recipients': await self.get_recipients() } # Update request status await self.update_request(request_id, 'completed', response) return response async def process_erasure_request(self, request_id: str) -> Dict: """Process a right to erasure request.""" request = await self.get_request(request_id) if request['type'] != 'erasure': raise ValueError("Not an erasure request") results = {} exceptions = [] for source in self.data_sources: try: # Check for legal exceptions can_delete, reason = await source.can_delete(request['user_id']) if can_delete: await source.delete_user_data(request['user_id']) results[source.name] = 'deleted' else: exceptions.append({ 'source': source.name, 'reason': reason # e.g., 'legal retention requirement' }) results[source.name] = f'retained: {reason}' except Exception as e: results[source.name] = f'error: {str(e)}' response = { 'request_id': request_id, 'completed_at': datetime.utcnow().isoformat(), 'results': results, 'exceptions': exceptions } await self.update_request(request_id, 'completed', response) return response async def process_portability_request(self, request_id: str) -> bytes: """Generate portable data export.""" request = await self.get_request(request_id) user_data = await self.process_access_request(request_id) # Convert to machine-readable format (JSON) portable_data = { 'export_date': datetime.utcnow().isoformat(), 'format_version': '1.0', 'data': user_data['data'] } return json.dumps(portable_data, indent=2, default=str).encode() ``` ### Pattern 3: Data Retention ```python from datetime import datetime, timedelta from enum import Enum class RetentionBasis(Enum): CONSENT = "consent" CONTRACT = "contract" LEGAL_OBLIGATION = "legal_obligation" LEGITIMATE_INTEREST = "legitimate_interest" class DataRetentionPolicy: """Define and enforce data retention policies.""" POLICIES = { 'user_account': { 'retention_period_days': 365 * 3, # 3 years after last activity 'basis': RetentionBasis.CONTRACT, 'trigger': 'last_activity_date', 'archive_before_delete': True }, 'transaction_records': { 'retention_period_days': 365 * 7, # 7 years for tax 'basis': RetentionBasis.LEGAL_OBLIGATION, 'trigger': 'transaction_date', 'archive_before_delete': True, 'legal_reference': 'Tax regulations require 7 year retention' }, 'marketing_consent': { 'retention_period_days': 365 * 2, # 2 years 'basis': RetentionBasis.CONSENT, 'trigger': 'consent_date', 'archive_before_delete': False }, 'support_tickets': { 'retention_period_days': 365 * 2, 'basis': RetentionBasis.LEGITIMATE_INTEREST, 'trigger': 'ticket_closed_date', 'archive_before_delete': True }, 'analytics_data': { 'retention_period_days': 365, # 1 year 'basis': RetentionBasis.CONSENT, 'trigger': 'collection_date', 'archive_before_delete': False, 'anonymize_instead': True } } async def apply_retention_policies(self): """Run retention policy enforcement.""" for data_type, policy in self.POLICIES.items(): cutoff_date = datetime.utcnow() - timedelta( days=policy['retention_period_days'] ) if policy.get('anonymize_instead'): await self.anonymize_old_data(data_type, cutoff_date) else: if policy.get('archive_before_delete'): await self.archive_data(data_type, cutoff_date) await self.delete_old_data(data_type, cutoff_date) await self.log_retention_action(data_type, cutoff_date) async def anonymize_old_data(self, data_type: str, before_date: datetime): """Anonymize data instead of deleting.""" # Example: Replace identifying fields with hashes if data_type == 'analytics_data': await self.db.analytics.update_many( {'collection_date': {'$lt': before_date}}, {'$set': { 'user_id': None, 'ip_address': None, 'device_id': None, 'anonymized': True, 'anonymized_date': datetime.utcnow() }} ) ``` ### Pattern 4: Privacy by Design ```python class PrivacyFirstDataModel: """Example of privacy-by-design data model.""" # Separate PII from behavioral data user_profile_schema = { 'user_id': str, # UUID, not sequential 'email_hash': str, # Hashed for lookups 'created_at': datetime, # Minimal data collection 'preferences': { 'language': str, 'timezone': str } } # Encrypted at rest user_pii_schema = { 'user_id': str, 'email': str, # Encrypted 'name': str, # Encrypted 'phone': str, # Encrypted (optional) 'address': dict, # Encrypted (optional) 'encryption_key_id': str } # Pseudonymized behavioral data analytics_schema = { 'session_id': str, # Not linked to user_id 'pseudonym_id': str, # Rotating pseudonym 'events': list, 'device_category': str, # Generalized, not specific 'country': str, # Not city-level } class DataMinimization: """Implement data minimization principles.""" @staticmethod def collect_only_needed(form_data: dict, purpose: str) -> dict: """Filter form data to only fields needed for purpose.""" REQUIRED_FIELDS = { 'account_creation': ['email', 'password'], 'newsletter': ['email'], 'purchase': ['email', 'name', 'address', 'payment'], 'support': ['email', 'message'] } allowed = REQUIRED_FIELDS.get(purpose, []) return {k: v for k, v in form_data.items() if k in allowed} @staticmethod def generalize_location(ip_address: str) -> str: """Generalize IP to country level only.""" import geoip2.database reader = geoip2.database.Reader('GeoLite2-Country.mmdb') try: response = reader.country(ip_address) return response.country.iso_code except: return 'UNKNOWN' ``` ### Pattern 5: Breach Notification ```python from datetime import datetime from enum import Enum class BreachSeverity(Enum): LOW = "low" MEDIUM = "medium" HIGH = "high" CRITICAL = "critical" class BreachNotificationHandler: """Handle GDPR breach notification requirements.""" AUTHORITY_NOTIFICATION_HOURS = 72 AFFECTED_NOTIFICATION_REQUIRED_SEVERITY = BreachSeverity.HIGH async def report_breach( self, description: str, data_types: List[str], affected_count: int, severity: BreachSeverity ) -> dict: """Report and handle a data breach.""" breach = { 'id': self.generate_breach_id(), 'reported_at': datetime.utcnow(), 'description': description, 'data_types_affected': data_types, 'affected_individuals_count': affected_count, 'severity': severity.value, 'status': 'investigating', 'timeline': [{ 'event': 'breach_reported', 'timestamp': datetime.utcnow(), 'details': description }] } await self.db.breaches.insert_one(breach) # Immediate notifications await self.notify_dpo(breach) await self.notify_security_team(breach) # Authority notification required within 72 hours if self.requires_authority_notification(severity, data_types): breach['authority_notification_deadline'] = ( datetime.utcnow() + timedelta(hours=self.AUTHORITY_NOTIFICATION_HOURS) ) await self.schedule_authority_notification(breach) # Affected individuals notification if severity.value in [BreachSeverity.HIGH.value, BreachSeverity.CRITICAL.value]: await self.schedule_individual_notifications(breach) return breach def requires_authority_notification( self, severity: BreachSeverity, data_types: List[str] ) -> bool: """Determine if supervisory authority must be notified.""" # Always notify for sensitive data sensitive_types = ['health', 'financial', 'credentials', 'biometric'] if any(t in sensitive_types for t in data_types): return True # Notify for medium+ severity return severity in [BreachSeverity.MEDIUM, BreachSeverity.HIGH, BreachSeverity.CRITICAL] async def generate_authority_report(self, breach_id: str) -> dict: """Generate report for supervisory authority.""" breach = await self.get_breach(breach_id) return { 'organization': { 'name': self.config.org_name, 'contact': self.config.dpo_contact, 'registration': self.config.registration_number }, 'breach': { 'nature': breach['description'], 'categories_affected': breach['data_types_affected'], 'approximate_number_affected': breach['affected_individuals_count'], 'likely_consequences': self.assess_consequences(breach), 'measures_taken': await self.get_remediation_measures(breach_id), 'measures_proposed': await self.get_proposed_measures(breach_id) }, 'timeline': breach['timeline'], 'submitted_at': datetime.utcnow().isoformat() } ``` ## Compliance Checklist ```markdown ## GDPR Implementation Checklist ### Legal Basis - [ ] Documented legal basis for each processing activity - [ ] Consent mechanisms meet GDPR requirements - [ ] Legitimate interest assessments completed ### Transparency - [ ] Privacy policy is clear and accessible - [ ] Processing purposes clearly stated - [ ] Data retention periods documented ### Data Subject Rights - [ ] Access request process implemented - [ ] Erasure request process implemented - [ ] Portability export available - [ ] Rectification process available - [ ] Response within 30-day deadline ### Security - [ ] Encryption at rest implemented - [ ] Encryption in transit (TLS) - [ ] Access controls in place - [ ] Audit logging enabled ### Breach Response - [ ] Breach detection mechanisms - [ ] 72-hour notification process - [ ] Breach documentation system ### Documentation - [ ] Records of processing activities (Art. 30) - [ ] Data protection impact assessments - [ ] Data processing agreements with vendors ``` ## Best Practices ### Do's - **Minimize data collection** - Only collect what's needed - **Document everything** - Processing activities, legal bases - **Encrypt PII** - At rest and in transit - **Implement access controls** - Need-to-know basis - **Regular audits** - Verify compliance continuously ### Don'ts - **Don't pre-check consent boxes** - Must be opt-in - **Don't bundle consent** - Separate purposes separately - **Don't retain indefinitely** - Defi
👍0
👁️0
🤖 Auto-discovered
🤖system prompt•7 months ago

javascript-testing-patterns

Implement comprehensive testing strategies using Jest, Vitest, and

coding
⭐1
# JavaScript Testing Patterns Comprehensive guide for implementing robust testing strategies in JavaScript/TypeScript applications using modern testing frameworks and best practices. ## When to Use This Skill - Setting up test infrastructure for new projects - Writing unit tests for functions and classes - Creating integration tests for APIs and services - Implementing end-to-end tests for user flows - Mocking external dependencies and APIs - Testing React, Vue, or other frontend components - Implementing test-driven development (TDD) - Setting up continuous testing in CI/CD pipelines ## Testing Frameworks ### Jest - Full-Featured Testing Framework **Setup:** ```typescript // jest.config.ts import type { Config } from "jest"; const config: Config = { preset: "ts-jest", testEnvironment: "node", roots: ["<rootDir>/src"], testMatch: ["**/__tests__/**/*.ts", "**/?(*.)+(spec|test).ts"], collectCoverageFrom: [ "src/**/*.ts", "!src/**/*.d.ts", "!src/**/*.interface.ts", ], coverageThreshold: { global: { branches: 80, functions: 80, lines: 80, statements: 80, }, }, setupFilesAfterEnv: ["<rootDir>/src/test/setup.ts"], }; export default config; ``` ### Vitest - Fast, Vite-Native Testing **Setup:** ```typescript // vitest.config.ts import { defineConfig } from "vitest/config"; export default defineConfig({ test: { globals: true, environment: "node", coverage: { provider: "v8", reporter: ["text", "json", "html"], exclude: ["**/*.d.ts", "**/*.config.ts", "**/dist/**"], }, setupFiles: ["./src/test/setup.ts"], }, }); ``` ## Unit Testing Patterns ### Pattern 1: Testing Pure Functions ```typescript // utils/calculator.ts export function add(a: number, b: number): number { return a + b; } export function divide(a: number, b: number): number { if (b === 0) { throw new Error("Division by zero"); } return a / b; } // utils/calculator.test.ts import { describe, it, expect } from "vitest"; import { add, divide } from "./calculator"; describe("Calculator", () => { describe("add", () => { it("should add two positive numbers", () => { expect(add(2, 3)).toBe(5); }); it("should add negative numbers", () => { expect(add(-2, -3)).toBe(-5); }); it("should handle zero", () => { expect(add(0, 5)).toBe(5); expect(add(5, 0)).toBe(5); }); }); describe("divide", () => { it("should divide two numbers", () => { expect(divide(10, 2)).toBe(5); }); it("should handle decimal results", () => { expect(divide(5, 2)).toBe(2.5); }); it("should throw error when dividing by zero", () => { expect(() => divide(10, 0)).toThrow("Division by zero"); }); }); }); ``` ### Pattern 2: Testing Classes ```typescript // services/user.service.ts export class UserService { private users: Map<string, User> = new Map(); create(user: User): User { if (this.users.has(user.id)) { throw new Error("User already exists"); } this.users.set(user.id, user); return user; } findById(id: string): User | undefined { return this.users.get(id); } update(id: string, updates: Partial<User>): User { const user = this.users.get(id); if (!user) { throw new Error("User not found"); } const updated = { ...user, ...updates }; this.users.set(id, updated); return updated; } delete(id: string): boolean { return this.users.delete(id); } } // services/user.service.test.ts import { describe, it, expect, beforeEach } from "vitest"; import { UserService } from "./user.service"; describe("UserService", () => { let service: UserService; beforeEach(() => { service = new UserService(); }); describe("create", () => { it("should create a new user", () => { const user = { id: "1", name: "John", email: "john@example.com" }; const created = service.create(user); expect(created).toEqual(user); expect(service.findById("1")).toEqual(user); }); it("should throw error if user already exists", () => { const user = { id: "1", name: "John", email: "john@example.com" }; service.create(user); expect(() => service.create(user)).toThrow("User already exists"); }); }); describe("update", () => { it("should update existing user", () => { const user = { id: "1", name: "John", email: "john@example.com" }; service.create(user); const updated = service.update("1", { name: "Jane" }); expect(updated.name).toBe("Jane"); expect(updated.email).toBe("john@example.com"); }); it("should throw error if user not found", () => { expect(() => service.update("999", { name: "Jane" })).toThrow( "User not found", ); }); }); }); ``` ### Pattern 3: Testing Async Functions ```typescript // services/api.service.ts export class ApiService { async fetchUser(id: string): Promise<User> { const response = await fetch(`https://api.example.com/users/${id}`); if (!response.ok) { throw new Error("User not found"); } return response.json(); } async createUser(user: CreateUserDTO): Promise<User> { const response = await fetch("https://api.example.com/users", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify(user), }); return response.json(); } } // services/api.service.test.ts import { describe, it, expect, vi, beforeEach } from "vitest"; import { ApiService } from "./api.service"; // Mock fetch globally global.fetch = vi.fn(); describe("ApiService", () => { let service: ApiService; beforeEach(() => { service = new ApiService(); vi.clearAllMocks(); }); describe("fetchUser", () => { it("should fetch user successfully", async () => { const mockUser = { id: "1", name: "John", email: "john@example.com" }; (fetch as any).mockResolvedValueOnce({ ok: true, json: async () => mockUser, }); const user = await service.fetchUser("1"); expect(user).toEqual(mockUser); expect(fetch).toHaveBeenCalledWith("https://api.example.com/users/1"); }); it("should throw error if user not found", async () => { (fetch as any).mockResolvedValueOnce({ ok: false, }); await expect(service.fetchUser("999")).rejects.toThrow("User not found"); }); }); describe("createUser", () => { it("should create user successfully", async () => { const newUser = { name: "John", email: "john@example.com" }; const createdUser = { id: "1", ...newUser }; (fetch as any).mockResolvedValueOnce({ ok: true, json: async () => createdUser, }); const user = await service.createUser(newUser); expect(user).toEqual(createdUser); expect(fetch).toHaveBeenCalledWith( "https://api.example.com/users", expect.objectContaining({ method: "POST", body: JSON.stringify(newUser), }), ); }); }); }); ``` ## Mocking Patterns ### Pattern 1: Mocking Modules ```typescript // services/email.service.ts import nodemailer from "nodemailer"; export class EmailService { private transporter = nodemailer.createTransport({ host: process.env.SMTP_HOST, port: 587, auth: { user: process.env.SMTP_USER, pass: process.env.SMTP_PASS, }, }); async sendEmail(to: string, subject: string, html: string) { await this.transporter.sendMail({ from: process.env.EMAIL_FROM, to, subject, html, }); } } // services/email.service.test.ts import { describe, it, expect, vi, beforeEach } from "vitest"; import { EmailService } from "./email.service"; vi.mock("nodemailer", () => ({ default: { createTransport: vi.fn(() => ({ sendMail: vi.fn().mockResolvedValue({ messageId: "123" }), })), }, })); describe("EmailService", () => { let service: EmailService; beforeEach(() => { service = new EmailService(); }); it("should send email successfully", async () => { await service.sendEmail( "test@example.com", "Test Subject", "<p>Test Body</p>", ); expect(service["transporter"].sendMail).toHaveBeenCalledWith( expect.objectContaining({ to: "test@example.com", subject: "Test Subject", }), ); }); }); ``` ### Pattern 2: Dependency Injection for Testing ```typescript // services/user.service.ts export interface IUserRepository { findById(id: string): Promise<User | null>; create(user: User): Promise<User>; } export class UserService { constructor(private userRepository: IUserRepository) {} async getUser(id: string): Promise<User> { const user = await this.userRepository.findById(id); if (!user) { throw new Error("User not found"); } return user; } async createUser(userData: CreateUserDTO): Promise<User> { // Business logic here const user = { id: generateId(), ...userData }; return this.userRepository.create(user); } } // services/user.service.test.ts import { describe, it, expect, vi, beforeEach } from "vitest"; import { UserService, IUserRepository } from "./user.service"; describe("UserService", () => { let service: UserService; let mockRepository: IUserRepository; beforeEach(() => { mockRepository = { findById: vi.fn(), create: vi.fn(), }; service = new UserService(mockRepository); }); describe("getUser", () => { it("should return user if found", async () => { const mockUser = { id: "1", name: "John", email: "john@example.com" }; vi.mocked(mockRepository.findById).mockResolvedValue(mockUser); const user = await service.getUser("1"); expect(user).toEqual(mockUser); expect(mockRepository.findById).toHaveBeenCalledWith("1"); }); it("should throw error if user not found", async () => { vi.mocked(mockRepository.findById).mockResolvedValue(null); await expect(service.getUser("999")).rejects.toThrow("User not found"); }); }); describe("createUser", () => { it("should create user successfully", async () => { const userData = { name: "John", email: "john@example.com" }; const createdUser = { id: "1", ...userData }; vi.mocked(mockRepository.create).mockResolvedValue(createdUser); const user = await service.createUser(userData); expect(user).toEqual(createdUser); expect(mockRepository.create).toHaveBeenCalled(); }); }); }); ``` ### Pattern 3: Spying on Functions ```typescript // utils/logger.ts export const logger = { info: (message: string) => console.log(`INFO: ${message}`), error: (message: string) => console.error(`ERROR: ${message}`), }; // services/order.service.ts import { logger } from "../utils/logger"; export class OrderService { async processOrder(orderId: string): Promise<void> { logger.info(`Processing order ${orderId}`); // Process order logic logger.info(`Order ${orderId} processed successfully`); } } // services/order.service.test.ts import { describe, it, expect, vi, beforeEach, afterEach } from "vitest"; import { OrderService } from "./order.service"; import { logger } from "../utils/logger"; describe("OrderService", () => { let service: OrderService; let loggerSpy: any; beforeEach(() => { service = new OrderService(); loggerSpy = vi.spyOn(logger, "info"); }); afterEach(() => { loggerSpy.mockRestore(); }); it("should log order processing", async () => { await service.processOrder("123"); expect(loggerSpy).toHaveBeenCalledWith("Processing order 123"); expect(loggerSpy).toHaveBeenCalledWith("Order 123 processed successfully"); expect(loggerSpy).toHaveBeenCalledTimes(2); }); }); ``` ## Integration Testing ### Pattern 1: API Integration Tests ```typescript // tests/integration/user.api.test.ts import request from "supertest"; import { app } from "../../src/app"; import { pool } from "../../src/config/database"; describe("User API Integration Tests", () => { beforeAll(async () => { // Setup test database await pool.query("CREATE TABLE IF NOT EXISTS users (...)"); }); afterAll(async () => { // Cleanup await pool.query("DROP TABLE IF EXISTS users"); await pool.end(); }); beforeEach(async () => { // Clear data before each test await pool.query("TRUNCATE TABLE users CASCADE"); }); describe("POST /api/users", () => { it("should create a new user", async () => { const userData = { name: "John Doe", email: "john@example.com", password: "password123", }; const response = await request(app) .post("/api/users") .send(userData) .expect(201); expect(response.body).toMatchObject({ name: userData.name, email: userData.email, }); expect(response.body).toHaveProperty("id"); expect(response.body).not.toHaveProperty("password"); }); it("should return 400 if email is invalid", async () => { const userData = { name: "John Doe", email: "invalid-email", password: "password123", }; const response = await request(app) .post("/api/users") .send(userData) .expect(400); expect(response.body).toHaveProperty("error"); }); it("should return 409 if email already exists", async () => { const userData = { name: "John Doe", email: "john@example.com", password: "password123", }; await request(app).post("/api/users").send(userData); const response = await request(app) .post("/api/users") .send(userData) .expect(409); expect(response.body.error).toContain("already exists"); }); }); describe("GET /api/users/:id", () => { it("should get user by id", async () => { const createResponse = await request(app).post("/api/users").send({ name: "John Doe", email: "john@example.com", password: "password123", }); const userId = createResponse.body.id; const response = await request(app) .get(`/api/users/${userId}`) .expect(200); expect(response.body).toMatchObject({ id: userId, name: "John Doe", email: "john@example.com", }); }); it("should return 404 if user not found", async () => { await request(app).get("/api/users/999").expect(404); }); }); describe("Authentication", () => { it("should require authentication for protected routes", async () => { await request(app).get("/api/users/me").expect(401); }); it("should allow access with valid token", async () => { // Create user and login await request(app).post("/api/users").send({ name: "John Doe", email: "john@example.com", password: "password123", }); const loginResponse = await request(app).post("/api/auth/login").send({ email: "john@example.com", password: "password123", }); const token = loginResponse.body.token; const response = await request(app) .get("/api/users/me") .set("Authorization", `Bearer ${token}`) .expect(200); expect(response.body.email).toBe("john@example.com"); }); }); }); ``` ### Pattern 2: Database Integration Tests ```typescript // tests/integration/user.repository.test.ts import { describe, it, expect, beforeAll, afterAll, beforeEach } from "vitest"; import { Pool } from "pg"; import { UserRepository } from "../../src/repositories/user.repository"; describe("UserRepository Integration Tests", () => { let pool: Pool; let repository: UserRepository; beforeAll(async () => { pool = new Pool({ host: "localhost", port: 5432, database: "test_db", user: "test_user", password: "test_password", }); repository = new UserRepository(pool); // Create tables await pool.query(` CREATE TABLE IF NOT EXISTS users ( id SERIAL PRIMARY KEY, name VARCHAR(255) NOT NULL, email VARCHAR(255) UNIQUE NOT NULL, password VARCHAR(255) NOT NULL, created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ) `); }); afterAll(async () => { await pool.query("DROP TABLE IF EXISTS users"); await pool.end(); }); beforeEach(async () => { await pool.query("TRUNCATE TABLE users CASCADE"); }); it("should create a user", async () => { const user = await repository.create({ name: "John Doe", email: "john@example.com", password: "hashed_password", }); expect(user).toHaveProperty("id"); expect(user.name).toBe("John Doe"); expect(user.email).toBe("john@example.com"); }); it("should find user by email", async () => { await repository.create({ name: "John Doe", email: "john@example.com", password: "hashed_password", }); const user = await repository.findByEmail("john@example.com"); expect(user).toBeTruthy(); expect(user?.name).toBe("John Doe"); }); it("should return null if user not found", async () => { const user = await repository.findByEmail("nonexistent@example.com"); expect(user).toBeNull(); }); }); ``` ## Frontend Testing with Testing Library ### Pattern 1: React Component Testing ```typescript // components/UserForm.tsx import { useState } from 'react'; interface Props { onSubmit: (user: { name: string; email: string }) => void; } export function UserForm({ onSubmit }: Props) { const [name, setName] = useState(''); const [email, setEmail] = useState(''); const handleSubmit = (e: React.FormEvent) => { e.preventDefault(); onSubmit({ name, email }); }; return ( <form onSubmit={handleSubmit}> <input type="text" placeholder="Name" value={name} onChange={(e) => setName(e.target.value)} data-testid="name-input" /> <input type="email" placeholder="Email" value={email} onChange={(e) => setEmail(e.target.value)} data-testid="email-input" /> <button type="submit">Submit</button> </form> ); } // components/UserForm.test.tsx import { render, screen, fireEvent } from '@testing-library/react'; import { describe, it, expect, vi } from 'vitest'; import { UserForm } from './UserForm'; describe('UserForm', () => { it('should render form inputs', () => { render(<UserForm onSubmit={vi.fn()} />); expect(screen.getByPlaceholderText('Name')).toBeInTheDocument(); expect(screen.getByPlaceholderText('Email')).toBeInTheDocument(); expect(screen.getByRole('button', { name: 'Submit' })).toBeInTheDocument(); }); it('should update input values', () => { render(<UserForm onSubmit={vi.fn()} />); const nameInput = screen.getByTestId('name-input') as HTMLInputElement; const emailInput = screen.getByTestId('email-input') as HTMLInputElement; fireEvent.change(nameInput, { target: { value: 'John Doe' } }); fireEvent.change(emailInpu
👍0
👁️0
🤖 Auto-discovered
🤖system prompt•7 months ago

modern-javascript-patterns

Master ES6+ features including async/await, destructuring, spread

coding
⭐1
# Modern JavaScript Patterns Comprehensive guide for mastering modern JavaScript (ES6+) features, functional programming patterns, and best practices for writing clean, maintainable, and performant code. ## When to Use This Skill - Refactoring legacy JavaScript to modern syntax - Implementing functional programming patterns - Optimizing JavaScript performance - Writing maintainable and readable code - Working with asynchronous operations - Building modern web applications - Migrating from callbacks to Promises/async-await - Implementing data transformation pipelines ## ES6+ Core Features ### 1. Arrow Functions **Syntax and Use Cases:** ```javascript // Traditional function function add(a, b) { return a + b; } // Arrow function const add = (a, b) => a + b; // Single parameter (parentheses optional) const double = (x) => x * 2; // No parameters const getRandom = () => Math.random(); // Multiple statements (need curly braces) const processUser = (user) => { const normalized = user.name.toLowerCase(); return { ...user, name: normalized }; }; // Returning objects (wrap in parentheses) const createUser = (name, age) => ({ name, age }); ``` **Lexical 'this' Binding:** ```javascript class Counter { constructor() { this.count = 0; } // Arrow function preserves 'this' context increment = () => { this.count++; }; // Traditional function loses 'this' in callbacks incrementTraditional() { setTimeout(function () { this.count++; // 'this' is undefined }, 1000); } // Arrow function maintains 'this' incrementArrow() { setTimeout(() => { this.count++; // 'this' refers to Counter instance }, 1000); } } ``` ### 2. Destructuring **Object Destructuring:** ```javascript const user = { id: 1, name: "John Doe", email: "john@example.com", address: { city: "New York", country: "USA", }, }; // Basic destructuring const { name, email } = user; // Rename variables const { name: userName, email: userEmail } = user; // Default values const { age = 25 } = user; // Nested destructuring const { address: { city, country }, } = user; // Rest operator const { id, ...userWithoutId } = user; // Function parameters function greet({ name, age = 18 }) { console.log(`Hello ${name}, you are ${age}`); } greet(user); ``` **Array Destructuring:** ```javascript const numbers = [1, 2, 3, 4, 5]; // Basic destructuring const [first, second] = numbers; // Skip elements const [, , third] = numbers; // Rest operator const [head, ...tail] = numbers; // Swapping variables let a = 1, b = 2; [a, b] = [b, a]; // Function return values function getCoordinates() { return [10, 20]; } const [x, y] = getCoordinates(); // Default values const [one, two, three = 0] = [1, 2]; ``` ### 3. Spread and Rest Operators **Spread Operator:** ```javascript // Array spreading const arr1 = [1, 2, 3]; const arr2 = [4, 5, 6]; const combined = [...arr1, ...arr2]; // Object spreading const defaults = { theme: "dark", lang: "en" }; const userPrefs = { theme: "light" }; const settings = { ...defaults, ...userPrefs }; // Function arguments const numbers = [1, 2, 3]; Math.max(...numbers); // Copying arrays/objects (shallow copy) const copy = [...arr1]; const objCopy = { ...user }; // Adding items immutably const newArr = [...arr1, 4, 5]; const newObj = { ...user, age: 30 }; ``` **Rest Parameters:** ```javascript // Collect function arguments function sum(...numbers) { return numbers.reduce((total, num) => total + num, 0); } sum(1, 2, 3, 4, 5); // With regular parameters function greet(greeting, ...names) { return `${greeting} ${names.join(", ")}`; } greet("Hello", "John", "Jane", "Bob"); // Object rest const { id, ...userData } = user; // Array rest const [first, ...rest] = [1, 2, 3, 4, 5]; ``` ### 4. Template Literals ```javascript // Basic usage const name = "John"; const greeting = `Hello, ${name}!`; // Multi-line strings const html = ` <div> <h1>${title}</h1> <p>${content}</p> </div> `; // Expression evaluation const price = 19.99; const total = `Total: $${(price * 1.2).toFixed(2)}`; // Tagged template literals function highlight(strings, ...values) { return strings.reduce((result, str, i) => { const value = values[i] || ""; return result + str + `<mark>${value}</mark>`; }, ""); } const name = "John"; const age = 30; const html = highlight`Name: ${name}, Age: ${age}`; // Output: "Name: <mark>John</mark>, Age: <mark>30</mark>" ``` ### 5. Enhanced Object Literals ```javascript const name = "John"; const age = 30; // Shorthand property names const user = { name, age }; // Shorthand method names const calculator = { add(a, b) { return a + b; }, subtract(a, b) { return a - b; }, }; // Computed property names const field = "email"; const user = { name: "John", [field]: "john@example.com", [`get${field.charAt(0).toUpperCase()}${field.slice(1)}`]() { return this[field]; }, }; // Dynamic property creation const createUser = (name, ...props) => { return props.reduce( (user, [key, value]) => ({ ...user, [key]: value, }), { name }, ); }; const user = createUser("John", ["age", 30], ["email", "john@example.com"]); ``` ## Asynchronous Patterns ### 1. Promises **Creating and Using Promises:** ```javascript // Creating a promise const fetchUser = (id) => { return new Promise((resolve, reject) => { setTimeout(() => { if (id > 0) { resolve({ id, name: "John" }); } else { reject(new Error("Invalid ID")); } }, 1000); }); }; // Using promises fetchUser(1) .then((user) => console.log(user)) .catch((error) => console.error(error)) .finally(() => console.log("Done")); // Chaining promises fetchUser(1) .then((user) => fetchUserPosts(user.id)) .then((posts) => processPosts(posts)) .then((result) => console.log(result)) .catch((error) => console.error(error)); ``` **Promise Combinators:** ```javascript // Promise.all - Wait for all promises const promises = [fetchUser(1), fetchUser(2), fetchUser(3)]; Promise.all(promises) .then((users) => console.log(users)) .catch((error) => console.error("At least one failed:", error)); // Promise.allSettled - Wait for all, regardless of outcome Promise.allSettled(promises).then((results) => { results.forEach((result) => { if (result.status === "fulfilled") { console.log("Success:", result.value); } else { console.log("Error:", result.reason); } }); }); // Promise.race - First to complete Promise.race(promises) .then((winner) => console.log("First:", winner)) .catch((error) => console.error(error)); // Promise.any - First to succeed Promise.any(promises) .then((first) => console.log("First success:", first)) .catch((error) => console.error("All failed:", error)); ``` ### 2. Async/Await **Basic Usage:** ```javascript // Async function always returns a Promise async function fetchUser(id) { const response = await fetch(`/api/users/${id}`); const user = await response.json(); return user; } // Error handling with try/catch async function getUserData(id) { try { const user = await fetchUser(id); const posts = await fetchUserPosts(user.id); return { user, posts }; } catch (error) { console.error("Error fetching data:", error); throw error; } } // Sequential vs Parallel execution async function sequential() { const user1 = await fetchUser(1); // Wait const user2 = await fetchUser(2); // Then wait return [user1, user2]; } async function parallel() { const [user1, user2] = await Promise.all([fetchUser(1), fetchUser(2)]); return [user1, user2]; } ``` **Advanced Patterns:** ```javascript // Async IIFE (async () => { const result = await someAsyncOperation(); console.log(result); })(); // Async iteration async function processUsers(userIds) { for (const id of userIds) { const user = await fetchUser(id); await processUser(user); } } // Top-level await (ES2022) const config = await fetch("/config.json").then((r) => r.json()); // Retry logic async function fetchWithRetry(url, retries = 3) { for (let i = 0; i < retries; i++) { try { return await fetch(url); } catch (error) { if (i === retries - 1) throw error; await new Promise((resolve) => setTimeout(resolve, 1000 * (i + 1))); } } } // Timeout wrapper async function withTimeout(promise, ms) { const timeout = new Promise((_, reject) => setTimeout(() => reject(new Error("Timeout")), ms), ); return Promise.race([promise, timeout]); } ``` ## Functional Programming Patterns ### 1. Array Methods **Map, Filter, Reduce:** ```javascript const users = [ { id: 1, name: "John", age: 30, active: true }, { id: 2, name: "Jane", age: 25, active: false }, { id: 3, name: "Bob", age: 35, active: true }, ]; // Map - Transform array const names = users.map((user) => user.name); const upperNames = users.map((user) => user.name.toUpperCase()); // Filter - Select elements const activeUsers = users.filter((user) => user.active); const adults = users.filter((user) => user.age >= 18); // Reduce - Aggregate data const totalAge = users.reduce((sum, user) => sum + user.age, 0); const avgAge = totalAge / users.length; // Group by property const byActive = users.reduce((groups, user) => { const key = user.active ? "active" : "inactive"; return { ...groups, [key]: [...(groups[key] || []), user], }; }, {}); // Chaining methods const result = users .filter((user) => user.active) .map((user) => user.name) .sort() .join(", "); ``` **Advanced Array Methods:** ```javascript // Find - First matching element const user = users.find((u) => u.id === 2); // FindIndex - Index of first match const index = users.findIndex((u) => u.name === "Jane"); // Some - At least one matches const hasActive = users.some((u) => u.active); // Every - All match const allAdults = users.every((u) => u.age >= 18); // FlatMap - Map and flatten const userTags = [ { name: "John", tags: ["admin", "user"] }, { name: "Jane", tags: ["user"] }, ]; const allTags = userTags.flatMap((u) => u.tags); // From - Create array from iterable const str = "hello"; const chars = Array.from(str); const numbers = Array.from({ length: 5 }, (_, i) => i + 1); // Of - Create array from arguments const arr = Array.of(1, 2, 3); ``` ### 2. Higher-Order Functions **Functions as Arguments:** ```javascript // Custom forEach function forEach(array, callback) { for (let i = 0; i < array.length; i++) { callback(array[i], i, array); } } // Custom map function map(array, transform) { const result = []; for (const item of array) { result.push(transform(item)); } return result; } // Custom filter function filter(array, predicate) { const result = []; for (const item of array) { if (predicate(item)) { result.push(item); } } return result; } ``` **Functions Returning Functions:** ```javascript // Currying const multiply = (a) => (b) => a * b; const double = multiply(2); const triple = multiply(3); console.log(double(5)); // 10 console.log(triple(5)); // 15 // Partial application function partial(fn, ...args) { return (...moreArgs) => fn(...args, ...moreArgs); } const add = (a, b, c) => a + b + c; const add5 = partial(add, 5); console.log(add5(3, 2)); // 10 // Memoization function memoize(fn) { const cache = new Map(); return (...args) => { const key = JSON.stringify(args); if (cache.has(key)) { return cache.get(key); } const result = fn(...args); cache.set(key, result); return result; }; } const fibonacci = memoize((n) => { if (n <= 1) return n; return fibonacci(n - 1) + fibonacci(n - 2); }); ``` ### 3. Composition and Piping ```javascript // Function composition const compose = (...fns) => (x) => fns.reduceRight((acc, fn) => fn(acc), x); const pipe = (...fns) => (x) => fns.reduce((acc, fn) => fn(acc), x); // Example usage const addOne = (x) => x + 1; const double = (x) => x * 2; const square = (x) => x * x; const composed = compose(square, double, addOne); console.log(composed(3)); // ((3 + 1) * 2)^2 = 64 const piped = pipe(addOne, double, square); console.log(piped(3)); // ((3 + 1) * 2)^2 = 64 // Practical example const processUser = pipe( (user) => ({ ...user, name: user.name.trim() }), (user) => ({ ...user, email: user.email.toLowerCase() }), (user) => ({ ...user, age: parseInt(user.age) }), ); const user = processUser({ name: " John ", email: "JOHN@EXAMPLE.COM", age: "30", }); ``` ### 4. Pure Functions and Immutability ```javascript // Impure function (modifies input) function addItemImpure(cart, item) { cart.items.push(item); cart.total += item.price; return cart; } // Pure function (no side effects) function addItemPure(cart, item) { return { ...cart, items: [...cart.items, item], total: cart.total + item.price, }; } // Immutable array operations const numbers = [1, 2, 3, 4, 5]; // Add to array const withSix = [...numbers, 6]; // Remove from array const withoutThree = numbers.filter((n) => n !== 3); // Update array element const doubled = numbers.map((n) => (n === 3 ? n * 2 : n)); // Immutable object operations const user = { name: "John", age: 30 }; // Update property const olderUser = { ...user, age: 31 }; // Add property const withEmail = { ...user, email: "john@example.com" }; // Remove property const { age, ...withoutAge } = user; // Deep cloning (simple approach) const deepClone = (obj) => JSON.parse(JSON.stringify(obj)); // Better deep cloning const structuredClone = (obj) => globalThis.structuredClone(obj); ``` ## Modern Class Features ```javascript // Class syntax class User { // Private fields #password; // Public fields id; name; // Static field static count = 0; constructor(id, name, password) { this.id = id; this.name = name; this.#password = password; User.count++; } // Public method greet() { return `Hello, ${this.name}`; } // Private method #hashPassword(password) { return `hashed_${password}`; } // Getter get displayName() { return this.name.toUpperCase(); } // Setter set password(newPassword) { this.#password = this.#hashPassword(newPassword); } // Static method static create(id, name, password) { return new User(id, name, password); } } // Inheritance class Admin extends User { constructor(id, name, password, role) { super(id, name, password); this.role = role; } greet() { return `${super.greet()}, I'm an admin`; } } ``` ## Modules (ES6) ```javascript // Exporting // math.js export const PI = 3.14159; export function add(a, b) { return a + b; } export class Calculator { // ... } // Default export export default function multiply(a, b) { return a * b; } // Importing // app.js import multiply, { PI, add, Calculator } from "./math.js"; // Rename imports import { add as sum } from "./math.js"; // Import all import * as Math from "./math.js"; // Dynamic imports const module = await import("./math.js"); const { add } = await import("./math.js"); // Conditional loading if (condition) { const module = await import("./feature.js"); module.init(); } ``` ## Iterators and Generators ```javascript // Custom iterator const range = { from: 1, to: 5, [Symbol.iterator]() { return { current: this.from, last: this.to, next() { if (this.current <= this.last) { return { done: false, value: this.current++ }; } else { return { done: true }; } }, }; }, }; for (const num of range) { console.log(num); // 1, 2, 3, 4, 5 } // Generator function function* rangeGenerator(from, to) { for (let i = from; i <= to; i++) { yield i; } } for (const num of rangeGenerator(1, 5)) { console.log(num); } // Infinite generator function* fibonacci() { let [prev, curr] = [0, 1]; while (true) { yield curr; [prev, curr] = [curr, prev + curr]; } } // Async generator async function* fetchPages(url) { let page = 1; while (true) { const response = await fetch(`${url}?page=${page}`); const data = await response.json(); if (data.length === 0) break; yield data; page++; } } for await (const page of fetchPages("/api/users")) { console.log(page); } ``` ## Modern Operators ```javascript // Optional chaining const user = { name: "John", address: { city: "NYC" } }; const city = user?.address?.city; const zipCode = user?.address?.zipCode; // undefined // Function call const result = obj.method?.(); // Array access const first = arr?.[0]; // Nullish coalescing const value = null ?? "default"; // 'default' const value = undefined ?? "default"; // 'default' const value = 0 ?? "default"; // 0 (not 'default') const value = "" ?? "default"; // '' (not 'default') // Logical assignment let a = null; a ??= "default"; // a = 'default' let b = 5; b ??= 10; // b = 5 (unchanged) let obj = { count: 0 }; obj.count ||= 1; // obj.count = 1 obj.count &&= 2; // obj.count = 2 ``` ## Performance Optimization ```javascript // Debounce function debounce(fn, delay) { let timeoutId; return (...args) => { clearTimeout(timeoutId); timeoutId = setTimeout(() => fn(...args), delay); }; } const searchDebounced = debounce(search, 300); // Throttle function throttle(fn, limit) { let inThrottle; return (...args) => { if (!inThrottle) { fn(...args); inThrottle = true; setTimeout(() => (inThrottle = false), limit); } }; } const scrollThrottled = throttle(handleScroll, 100); // Lazy evaluation function* lazyMap(iterable, transform) { for (const item of iterable) { yield transform(item); } } // Use only what you need const numbers = [1, 2, 3, 4, 5]; const doubled = lazyMap(numbers, (x) => x * 2); const first = doubled.next().value; // Only computes first value ``` ## Best Practices 1. **Use const by default**: Only use let when reassignment is needed 2. **Prefer arrow functions**: Especially for callbacks 3. **Use template literals**: Instead of string concatenation 4. **Destructure objects and arrays**: For cleaner code 5. **Use async/await**: Instead of Promise chains 6. **Avoid mutating data**: Use spread operator and array methods 7. **Use optional chaining**: Prevent "Cannot read property of undefined" 8. **Use nullish coalescing**: For default values 9. **Prefer array methods**: Over traditional loops 10. **Use modules**: For better code organization 11. **Write pure functions**: Easier to test and reason about 12. **Use meaningful variable names**: Self-documenting code 13. **Keep functions small**: Single responsibility principle 14. **Handle errors properly**: Use try/catch with async/await 15. **Use strict mode**: `'use strict'` for better error catching ## Common Pitfalls 1. **this binding confusion**: Use arrow functions or bind() 2. **Async/await without error handling**: Always use try/catch 3. **Promise creation
👍0
👁️0
🤖 Auto-discovered
🤖system prompt•7 months ago

typescript-advanced-types

Master TypeScript's advanced type system including generics,

coding
⭐1
# TypeScript Advanced Types Comprehensive guidance for mastering TypeScript's advanced type system including generics, conditional types, mapped types, template literal types, and utility types for building robust, type-safe applications. ## When to Use This Skill - Building type-safe libraries or frameworks - Creating reusable generic components - Implementing complex type inference logic - Designing type-safe API clients - Building form validation systems - Creating strongly-typed configuration objects - Implementing type-safe state management - Migrating JavaScript codebases to TypeScript ## Core Concepts ### 1. Generics **Purpose:** Create reusable, type-flexible components while maintaining type safety. **Basic Generic Function:** ```typescript function identity<T>(value: T): T { return value; } const num = identity<number>(42); // Type: number const str = identity<string>("hello"); // Type: string const auto = identity(true); // Type inferred: boolean ``` **Generic Constraints:** ```typescript interface HasLength { length: number; } function logLength<T extends HasLength>(item: T): T { console.log(item.length); return item; } logLength("hello"); // OK: string has length logLength([1, 2, 3]); // OK: array has length logLength({ length: 10 }); // OK: object has length // logLength(42); // Error: number has no length ``` **Multiple Type Parameters:** ```typescript function merge<T, U>(obj1: T, obj2: U): T & U { return { ...obj1, ...obj2 }; } const merged = merge({ name: "John" }, { age: 30 }); // Type: { name: string } & { age: number } ``` ### 2. Conditional Types **Purpose:** Create types that depend on conditions, enabling sophisticated type logic. **Basic Conditional Type:** ```typescript type IsString<T> = T extends string ? true : false; type A = IsString<string>; // true type B = IsString<number>; // false ``` **Extracting Return Types:** ```typescript type ReturnType<T> = T extends (...args: any[]) => infer R ? R : never; function getUser() { return { id: 1, name: "John" }; } type User = ReturnType<typeof getUser>; // Type: { id: number; name: string; } ``` **Distributive Conditional Types:** ```typescript type ToArray<T> = T extends any ? T[] : never; type StrOrNumArray = ToArray<string | number>; // Type: string[] | number[] ``` **Nested Conditions:** ```typescript type TypeName<T> = T extends string ? "string" : T extends number ? "number" : T extends boolean ? "boolean" : T extends undefined ? "undefined" : T extends Function ? "function" : "object"; type T1 = TypeName<string>; // "string" type T2 = TypeName<() => void>; // "function" ``` ### 3. Mapped Types **Purpose:** Transform existing types by iterating over their properties. **Basic Mapped Type:** ```typescript type Readonly<T> = { readonly [P in keyof T]: T[P]; }; interface User { id: number; name: string; } type ReadonlyUser = Readonly<User>; // Type: { readonly id: number; readonly name: string; } ``` **Optional Properties:** ```typescript type Partial<T> = { [P in keyof T]?: T[P]; }; type PartialUser = Partial<User>; // Type: { id?: number; name?: string; } ``` **Key Remapping:** ```typescript type Getters<T> = { [K in keyof T as `get${Capitalize<string & K>}`]: () => T[K]; }; interface Person { name: string; age: number; } type PersonGetters = Getters<Person>; // Type: { getName: () => string; getAge: () => number; } ``` **Filtering Properties:** ```typescript type PickByType<T, U> = { [K in keyof T as T[K] extends U ? K : never]: T[K]; }; interface Mixed { id: number; name: string; age: number; active: boolean; } type OnlyNumbers = PickByType<Mixed, number>; // Type: { id: number; age: number; } ``` ### 4. Template Literal Types **Purpose:** Create string-based types with pattern matching and transformation. **Basic Template Literal:** ```typescript type EventName = "click" | "focus" | "blur"; type EventHandler = `on${Capitalize<EventName>}`; // Type: "onClick" | "onFocus" | "onBlur" ``` **String Manipulation:** ```typescript type UppercaseGreeting = Uppercase<"hello">; // "HELLO" type LowercaseGreeting = Lowercase<"HELLO">; // "hello" type CapitalizedName = Capitalize<"john">; // "John" type UncapitalizedName = Uncapitalize<"John">; // "john" ``` **Path Building:** ```typescript type Path<T> = T extends object ? { [K in keyof T]: K extends string ? `${K}` | `${K}.${Path<T[K]>}` : never; }[keyof T] : never; interface Config { server: { host: string; port: number; }; database: { url: string; }; } type ConfigPath = Path<Config>; // Type: "server" | "database" | "server.host" | "server.port" | "database.url" ``` ### 5. Utility Types **Built-in Utility Types:** ```typescript // Partial<T> - Make all properties optional type PartialUser = Partial<User>; // Required<T> - Make all properties required type RequiredUser = Required<PartialUser>; // Readonly<T> - Make all properties readonly type ReadonlyUser = Readonly<User>; // Pick<T, K> - Select specific properties type UserName = Pick<User, "name" | "email">; // Omit<T, K> - Remove specific properties type UserWithoutPassword = Omit<User, "password">; // Exclude<T, U> - Exclude types from union type T1 = Exclude<"a" | "b" | "c", "a">; // "b" | "c" // Extract<T, U> - Extract types from union type T2 = Extract<"a" | "b" | "c", "a" | "b">; // "a" | "b" // NonNullable<T> - Exclude null and undefined type T3 = NonNullable<string | null | undefined>; // string // Record<K, T> - Create object type with keys K and values T type PageInfo = Record<"home" | "about", { title: string }>; ``` ## Advanced Patterns ### Pattern 1: Type-Safe Event Emitter ```typescript type EventMap = { "user:created": { id: string; name: string }; "user:updated": { id: string }; "user:deleted": { id: string }; }; class TypedEventEmitter<T extends Record<string, any>> { private listeners: { [K in keyof T]?: Array<(data: T[K]) => void>; } = {}; on<K extends keyof T>(event: K, callback: (data: T[K]) => void): void { if (!this.listeners[event]) { this.listeners[event] = []; } this.listeners[event]!.push(callback); } emit<K extends keyof T>(event: K, data: T[K]): void { const callbacks = this.listeners[event]; if (callbacks) { callbacks.forEach((callback) => callback(data)); } } } const emitter = new TypedEventEmitter<EventMap>(); emitter.on("user:created", (data) => { console.log(data.id, data.name); // Type-safe! }); emitter.emit("user:created", { id: "1", name: "John" }); // emitter.emit("user:created", { id: "1" }); // Error: missing 'name' ``` ### Pattern 2: Type-Safe API Client ```typescript type HTTPMethod = "GET" | "POST" | "PUT" | "DELETE"; type EndpointConfig = { "/users": { GET: { response: User[] }; POST: { body: { name: string; email: string }; response: User }; }; "/users/:id": { GET: { params: { id: string }; response: User }; PUT: { params: { id: string }; body: Partial<User>; response: User }; DELETE: { params: { id: string }; response: void }; }; }; type ExtractParams<T> = T extends { params: infer P } ? P : never; type ExtractBody<T> = T extends { body: infer B } ? B : never; type ExtractResponse<T> = T extends { response: infer R } ? R : never; class APIClient<Config extends Record<string, Record<HTTPMethod, any>>> { async request<Path extends keyof Config, Method extends keyof Config[Path]>( path: Path, method: Method, ...[options]: ExtractParams<Config[Path][Method]> extends never ? ExtractBody<Config[Path][Method]> extends never ? [] : [{ body: ExtractBody<Config[Path][Method]> }] : [ { params: ExtractParams<Config[Path][Method]>; body?: ExtractBody<Config[Path][Method]>; }, ] ): Promise<ExtractResponse<Config[Path][Method]>> { // Implementation here return {} as any; } } const api = new APIClient<EndpointConfig>(); // Type-safe API calls const users = await api.request("/users", "GET"); // Type: User[] const newUser = await api.request("/users", "POST", { body: { name: "John", email: "john@example.com" }, }); // Type: User const user = await api.request("/users/:id", "GET", { params: { id: "123" }, }); // Type: User ``` ### Pattern 3: Builder Pattern with Type Safety ```typescript type BuilderState<T> = { [K in keyof T]: T[K] | undefined; }; type RequiredKeys<T> = { [K in keyof T]-?: {} extends Pick<T, K> ? never : K; }[keyof T]; type OptionalKeys<T> = { [K in keyof T]-?: {} extends Pick<T, K> ? K : never; }[keyof T]; type IsComplete<T, S> = RequiredKeys<T> extends keyof S ? S[RequiredKeys<T>] extends undefined ? false : true : false; class Builder<T, S extends BuilderState<T> = {}> { private state: S = {} as S; set<K extends keyof T>(key: K, value: T[K]): Builder<T, S & Record<K, T[K]>> { this.state[key] = value; return this as any; } build(this: IsComplete<T, S> extends true ? this : never): T { return this.state as T; } } interface User { id: string; name: string; email: string; age?: number; } const builder = new Builder<User>(); const user = builder .set("id", "1") .set("name", "John") .set("email", "john@example.com") .build(); // OK: all required fields set // const incomplete = builder // .set("id", "1") // .build(); // Error: missing required fields ``` ### Pattern 4: Deep Readonly/Partial ```typescript type DeepReadonly<T> = { readonly [P in keyof T]: T[P] extends object ? T[P] extends Function ? T[P] : DeepReadonly<T[P]> : T[P]; }; type DeepPartial<T> = { [P in keyof T]?: T[P] extends object ? T[P] extends Array<infer U> ? Array<DeepPartial<U>> : DeepPartial<T[P]> : T[P]; }; interface Config { server: { host: string; port: number; ssl: { enabled: boolean; cert: string; }; }; database: { url: string; pool: { min: number; max: number; }; }; } type ReadonlyConfig = DeepReadonly<Config>; // All nested properties are readonly type PartialConfig = DeepPartial<Config>; // All nested properties are optional ``` ### Pattern 5: Type-Safe Form Validation ```typescript type ValidationRule<T> = { validate: (value: T) => boolean; message: string; }; type FieldValidation<T> = { [K in keyof T]?: ValidationRule<T[K]>[]; }; type ValidationErrors<T> = { [K in keyof T]?: string[]; }; class FormValidator<T extends Record<string, any>> { constructor(private rules: FieldValidation<T>) {} validate(data: T): ValidationErrors<T> | null { const errors: ValidationErrors<T> = {}; let hasErrors = false; for (const key in this.rules) { const fieldRules = this.rules[key]; const value = data[key]; if (fieldRules) { const fieldErrors: string[] = []; for (const rule of fieldRules) { if (!rule.validate(value)) { fieldErrors.push(rule.message); } } if (fieldErrors.length > 0) { errors[key] = fieldErrors; hasErrors = true; } } } return hasErrors ? errors : null; } } interface LoginForm { email: string; password: string; } const validator = new FormValidator<LoginForm>({ email: [ { validate: (v) => v.includes("@"), message: "Email must contain @", }, { validate: (v) => v.length > 0, message: "Email is required", }, ], password: [ { validate: (v) => v.length >= 8, message: "Password must be at least 8 characters", }, ], }); const errors = validator.validate({ email: "invalid", password: "short", }); // Type: { email?: string[]; password?: string[]; } | null ``` ### Pattern 6: Discriminated Unions ```typescript type Success<T> = { status: "success"; data: T; }; type Error = { status: "error"; error: string; }; type Loading = { status: "loading"; }; type AsyncState<T> = Success<T> | Error | Loading; function handleState<T>(state: AsyncState<T>): void { switch (state.status) { case "success": console.log(state.data); // Type: T break; case "error": console.log(state.error); // Type: string break; case "loading": console.log("Loading..."); break; } } // Type-safe state machine type State = | { type: "idle" } | { type: "fetching"; requestId: string } | { type: "success"; data: any } | { type: "error"; error: Error }; type Event = | { type: "FETCH"; requestId: string } | { type: "SUCCESS"; data: any } | { type: "ERROR"; error: Error } | { type: "RESET" }; function reducer(state: State, event: Event): State { switch (state.type) { case "idle": return event.type === "FETCH" ? { type: "fetching", requestId: event.requestId } : state; case "fetching": if (event.type === "SUCCESS") { return { type: "success", data: event.data }; } if (event.type === "ERROR") { return { type: "error", error: event.error }; } return state; case "success": case "error": return event.type === "RESET" ? { type: "idle" } : state; } } ``` ## Type Inference Techniques ### 1. Infer Keyword ```typescript // Extract array element type type ElementType<T> = T extends (infer U)[] ? U : never; type NumArray = number[]; type Num = ElementType<NumArray>; // number // Extract promise type type PromiseType<T> = T extends Promise<infer U> ? U : never; type AsyncNum = PromiseType<Promise<number>>; // number // Extract function parameters type Parameters<T> = T extends (...args: infer P) => any ? P : never; function foo(a: string, b: number) {} type FooParams = Parameters<typeof foo>; // [string, number] ``` ### 2. Type Guards ```typescript function isString(value: unknown): value is string { return typeof value === "string"; } function isArrayOf<T>( value: unknown, guard: (item: unknown) => item is T, ): value is T[] { return Array.isArray(value) && value.every(guard); } const data: unknown = ["a", "b", "c"]; if (isArrayOf(data, isString)) { data.forEach((s) => s.toUpperCase()); // Type: string[] } ``` ### 3. Assertion Functions ```typescript function assertIsString(value: unknown): asserts value is string { if (typeof value !== "string") { throw new Error("Not a string"); } } function processValue(value: unknown) { assertIsString(value); // value is now typed as string console.log(value.toUpperCase()); } ``` ## Best Practices 1. **Use `unknown` over `any`**: Enforce type checking 2. **Prefer `interface` for object shapes**: Better error messages 3. **Use `type` for unions and complex types**: More flexible 4. **Leverage type inference**: Let TypeScript infer when possible 5. **Create helper types**: Build reusable type utilities 6. **Use const assertions**: Preserve literal types 7. **Avoid type assertions**: Use type guards instead 8. **Document complex types**: Add JSDoc comments 9. **Use strict mode**: Enable all strict compiler options 10. **Test your types**: Use type tests to verify type behavior ## Type Testing ```typescript // Type assertion tests type AssertEqual<T, U> = [T] extends [U] ? [U] extends [T] ? true : false : false; type Test1 = AssertEqual<string, string>; // true type Test2 = AssertEqual<string, number>; // false type Test3 = AssertEqual<string | number, string>; // false // Expect error helper type ExpectError<T extends never> = T; // Example usage type ShouldError = ExpectError<AssertEqual<string, number>>; ``` ## Common Pitfalls 1. **Over-using `any`**: Defeats the purpose of TypeScript 2. **Ignoring strict null checks**: Can lead to runtime errors 3. **Too complex types**: Can slow down compilation 4. **Not using discriminated unions**: Misses type narrowing opportunities 5. **Forgetting readonly modifiers**: Allows unintended mutations 6. **Circular type references**: Can cause compiler errors 7. **Not handling edge cases**: Like empty arrays or null values ## Performance Considerations - Avoid deeply nested conditional types - Use simple types when possible - Cache complex type computations - Limit recursion depth in recursive types - Use build tools to skip type checking in production ## Resources - **TypeScript Handbook**: https://www.typescriptlang.org/docs/handbook/ - **Type Challenges**: https://github.com/type-challenges/type-challenges - **TypeScript Deep Dive**: https://basarat.gitbook.io/typescript/ - **Effective TypeScript**: Book by Dan Vanderkam
👍0
👁️0
🤖 Auto-discovered
🤖system prompt•7 months ago

distributed-tracing

Implement distributed tracing with Jaeger and Tempo to track

coding
⭐1
# Distributed Tracing Implement distributed tracing with Jaeger and Tempo for request flow visibility across microservices. ## Purpose Track requests across distributed systems to understand latency, dependencies, and failure points. ## When to Use - Debug latency issues - Understand service dependencies - Identify bottlenecks - Trace error propagation - Analyze request paths ## Distributed Tracing Concepts ### Trace Structure ``` Trace (Request ID: abc123) ↓ Span (frontend) [100ms] ↓ Span (api-gateway) [80ms] ├→ Span (auth-service) [10ms] └→ Span (user-service) [60ms] └→ Span (database) [40ms] ``` ### Key Components - **Trace** - End-to-end request journey - **Span** - Single operation within a trace - **Context** - Metadata propagated between services - **Tags** - Key-value pairs for filtering - **Logs** - Timestamped events within a span ## Jaeger Setup ### Kubernetes Deployment ```bash # Deploy Jaeger Operator kubectl create namespace observability kubectl create -f https://github.com/jaegertracing/jaeger-operator/releases/download/v1.51.0/jaeger-operator.yaml -n observability # Deploy Jaeger instance kubectl apply -f - <<EOF apiVersion: jaegertracing.io/v1 kind: Jaeger metadata: name: jaeger namespace: observability spec: strategy: production storage: type: elasticsearch options: es: server-urls: http://elasticsearch:9200 ingress: enabled: true EOF ``` ### Docker Compose ```yaml version: "3.8" services: jaeger: image: jaegertracing/all-in-one:latest ports: - "5775:5775/udp" - "6831:6831/udp" - "6832:6832/udp" - "5778:5778" - "16686:16686" # UI - "14268:14268" # Collector - "14250:14250" # gRPC - "9411:9411" # Zipkin environment: - COLLECTOR_ZIPKIN_HOST_PORT=:9411 ``` **Reference:** See `references/jaeger-setup.md` ## Application Instrumentation ### OpenTelemetry (Recommended) #### Python (Flask) ```python from opentelemetry import trace from opentelemetry.exporter.jaeger.thrift import JaegerExporter from opentelemetry.sdk.resources import SERVICE_NAME, Resource from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import BatchSpanProcessor from opentelemetry.instrumentation.flask import FlaskInstrumentor from flask import Flask # Initialize tracer resource = Resource(attributes={SERVICE_NAME: "my-service"}) provider = TracerProvider(resource=resource) processor = BatchSpanProcessor(JaegerExporter( agent_host_name="jaeger", agent_port=6831, )) provider.add_span_processor(processor) trace.set_tracer_provider(provider) # Instrument Flask app = Flask(__name__) FlaskInstrumentor().instrument_app(app) @app.route('/api/users') def get_users(): tracer = trace.get_tracer(__name__) with tracer.start_as_current_span("get_users") as span: span.set_attribute("user.count", 100) # Business logic users = fetch_users_from_db() return {"users": users} def fetch_users_from_db(): tracer = trace.get_tracer(__name__) with tracer.start_as_current_span("database_query") as span: span.set_attribute("db.system", "postgresql") span.set_attribute("db.statement", "SELECT * FROM users") # Database query return query_database() ``` #### Node.js (Express) ```javascript const { NodeTracerProvider } = require("@opentelemetry/sdk-trace-node"); const { JaegerExporter } = require("@opentelemetry/exporter-jaeger"); const { BatchSpanProcessor } = require("@opentelemetry/sdk-trace-base"); const { registerInstrumentations } = require("@opentelemetry/instrumentation"); const { HttpInstrumentation } = require("@opentelemetry/instrumentation-http"); const { ExpressInstrumentation, } = require("@opentelemetry/instrumentation-express"); // Initialize tracer const provider = new NodeTracerProvider({ resource: { attributes: { "service.name": "my-service" } }, }); const exporter = new JaegerExporter({ endpoint: "http://jaeger:14268/api/traces", }); provider.addSpanProcessor(new BatchSpanProcessor(exporter)); provider.register(); // Instrument libraries registerInstrumentations({ instrumentations: [new HttpInstrumentation(), new ExpressInstrumentation()], }); const express = require("express"); const app = express(); app.get("/api/users", async (req, res) => { const tracer = trace.getTracer("my-service"); const span = tracer.startSpan("get_users"); try { const users = await fetchUsers(); span.setAttributes({ "user.count": users.length }); res.json({ users }); } finally { span.end(); } }); ``` #### Go ```go package main import ( "context" "go.opentelemetry.io/otel" "go.opentelemetry.io/otel/exporters/jaeger" "go.opentelemetry.io/otel/sdk/resource" sdktrace "go.opentelemetry.io/otel/sdk/trace" semconv "go.opentelemetry.io/otel/semconv/v1.4.0" ) func initTracer() (*sdktrace.TracerProvider, error) { exporter, err := jaeger.New(jaeger.WithCollectorEndpoint( jaeger.WithEndpoint("http://jaeger:14268/api/traces"), )) if err != nil { return nil, err } tp := sdktrace.NewTracerProvider( sdktrace.WithBatcher(exporter), sdktrace.WithResource(resource.NewWithAttributes( semconv.SchemaURL, semconv.ServiceNameKey.String("my-service"), )), ) otel.SetTracerProvider(tp) return tp, nil } func getUsers(ctx context.Context) ([]User, error) { tracer := otel.Tracer("my-service") ctx, span := tracer.Start(ctx, "get_users") defer span.End() span.SetAttributes(attribute.String("user.filter", "active")) users, err := fetchUsersFromDB(ctx) if err != nil { span.RecordError(err) return nil, err } span.SetAttributes(attribute.Int("user.count", len(users))) return users, nil } ``` **Reference:** See `references/instrumentation.md` ## Context Propagation ### HTTP Headers ``` traceparent: 00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01 tracestate: congo=t61rcWkgMzE ``` ### Propagation in HTTP Requests #### Python ```python from opentelemetry.propagate import inject headers = {} inject(headers) # Injects trace context response = requests.get('http://downstream-service/api', headers=headers) ``` #### Node.js ```javascript const { propagation } = require("@opentelemetry/api"); const headers = {}; propagation.inject(context.active(), headers); axios.get("http://downstream-service/api", { headers }); ``` ## Tempo Setup (Grafana) ### Kubernetes Deployment ```yaml apiVersion: v1 kind: ConfigMap metadata: name: tempo-config data: tempo.yaml: | server: http_listen_port: 3200 distributor: receivers: jaeger: protocols: thrift_http: grpc: otlp: protocols: http: grpc: storage: trace: backend: s3 s3: bucket: tempo-traces endpoint: s3.amazonaws.com querier: frontend_worker: frontend_address: tempo-query-frontend:9095 --- apiVersion: apps/v1 kind: Deployment metadata: name: tempo spec: replicas: 1 template: spec: containers: - name: tempo image: grafana/tempo:latest args: - -config.file=/etc/tempo/tempo.yaml volumeMounts: - name: config mountPath: /etc/tempo volumes: - name: config configMap: name: tempo-config ``` **Reference:** See `assets/jaeger-config.yaml.template` ## Sampling Strategies ### Probabilistic Sampling ```yaml # Sample 1% of traces sampler: type: probabilistic param: 0.01 ``` ### Rate Limiting Sampling ```yaml # Sample max 100 traces per second sampler: type: ratelimiting param: 100 ``` ### Adaptive Sampling ```python from opentelemetry.sdk.trace.sampling import ParentBased, TraceIdRatioBased # Sample based on trace ID (deterministic) sampler = ParentBased(root=TraceIdRatioBased(0.01)) ``` ## Trace Analysis ### Finding Slow Requests **Jaeger Query:** ``` service=my-service duration > 1s ``` ### Finding Errors **Jaeger Query:** ``` service=my-service error=true tags.http.status_code >= 500 ``` ### Service Dependency Graph Jaeger automatically generates service dependency graphs showing: - Service relationships - Request rates - Error rates - Average latencies ## Best Practices 1. **Sample appropriately** (1-10% in production) 2. **Add meaningful tags** (user_id, request_id) 3. **Propagate context** across all service boundaries 4. **Log exceptions** in spans 5. **Use consistent naming** for operations 6. **Monitor tracing overhead** (<1% CPU impact) 7. **Set up alerts** for trace errors 8. **Implement distributed context** (baggage) 9. **Use span events** for important milestones 10. **Document instrumentation** standards ## Integration with Logging ### Correlated Logs ```python import logging from opentelemetry import trace logger = logging.getLogger(__name__) def process_request(): span = trace.get_current_span() trace_id = span.get_span_context().trace_id logger.info( "Processing request", extra={"trace_id": format(trace_id, '032x')} ) ``` ## Troubleshooting **No traces appearing:** - Check collector endpoint - Verify network connectivity - Check sampling configuration - Review application logs **High latency overhead:** - Reduce sampling rate - Use batch span processor - Check exporter configuration ## Reference Files - `references/jaeger-setup.md` - Jaeger installation - `references/instrumentation.md` - Instrumentation patterns - `assets/jaeger-config.yaml.template` - Jaeger configuration ## Related Skills - `prometheus-configuration` - For metrics - `grafana-dashboards` - For visualization - `slo-implementation` - For latency SLOs
👍0
👁️0
🤖 Auto-discovered
🤖system prompt•7 months ago

paypal-integration

Integrate PayPal payment processing with support for express

business
⭐1
# PayPal Integration Master PayPal payment integration including Express Checkout, IPN handling, recurring billing, and refund workflows. ## When to Use This Skill - Integrating PayPal as a payment option - Implementing express checkout flows - Setting up recurring billing with PayPal - Processing refunds and payment disputes - Handling PayPal webhooks (IPN) - Supporting international payments - Implementing PayPal subscriptions ## Core Concepts ### 1. Payment Products **PayPal Checkout** - One-time payments - Express checkout experience - Guest and PayPal account payments **PayPal Subscriptions** - Recurring billing - Subscription plans - Automatic renewals **PayPal Payouts** - Send money to multiple recipients - Marketplace and platform payments ### 2. Integration Methods **Client-Side (JavaScript SDK)** - Smart Payment Buttons - Hosted payment flow - Minimal backend code **Server-Side (REST API)** - Full control over payment flow - Custom checkout UI - Advanced features ### 3. IPN (Instant Payment Notification) - Webhook-like payment notifications - Asynchronous payment updates - Verification required ## Quick Start ```javascript // Frontend - PayPal Smart Buttons <div id="paypal-button-container"></div> <script src="https://www.paypal.com/sdk/js?client-id=YOUR_CLIENT_ID&currency=USD"></script> <script> paypal.Buttons({ createOrder: function(data, actions) { return actions.order.create({ purchase_units: [{ amount: { value: '25.00' } }] }); }, onApprove: function(data, actions) { return actions.order.capture().then(function(details) { // Payment successful console.log('Transaction completed by ' + details.payer.name.given_name); // Send to backend for verification fetch('/api/paypal/capture', { method: 'POST', headers: {'Content-Type': 'application/json'}, body: JSON.stringify({orderID: data.orderID}) }); }); } }).render('#paypal-button-container'); </script> ``` ```python # Backend - Verify and capture order from paypalrestsdk import Payment import paypalrestsdk paypalrestsdk.configure({ "mode": "sandbox", # or "live" "client_id": "YOUR_CLIENT_ID", "client_secret": "YOUR_CLIENT_SECRET" }) def capture_paypal_order(order_id): """Capture a PayPal order.""" payment = Payment.find(order_id) if payment.execute({"payer_id": payment.payer.payer_info.payer_id}): # Payment successful return { 'status': 'success', 'transaction_id': payment.id, 'amount': payment.transactions[0].amount.total } else: # Payment failed return { 'status': 'failed', 'error': payment.error } ``` ## Express Checkout Implementation ### Server-Side Order Creation ```python import requests import json class PayPalClient: def __init__(self, client_id, client_secret, mode='sandbox'): self.client_id = client_id self.client_secret = client_secret self.base_url = 'https://api-m.sandbox.paypal.com' if mode == 'sandbox' else 'https://api-m.paypal.com' self.access_token = self.get_access_token() def get_access_token(self): """Get OAuth access token.""" url = f"{self.base_url}/v1/oauth2/token" headers = {"Accept": "application/json", "Accept-Language": "en_US"} response = requests.post( url, headers=headers, data={"grant_type": "client_credentials"}, auth=(self.client_id, self.client_secret) ) return response.json()['access_token'] def create_order(self, amount, currency='USD'): """Create a PayPal order.""" url = f"{self.base_url}/v2/checkout/orders" headers = { "Content-Type": "application/json", "Authorization": f"Bearer {self.access_token}" } payload = { "intent": "CAPTURE", "purchase_units": [{ "amount": { "currency_code": currency, "value": str(amount) } }] } response = requests.post(url, headers=headers, json=payload) return response.json() def capture_order(self, order_id): """Capture payment for an order.""" url = f"{self.base_url}/v2/checkout/orders/{order_id}/capture" headers = { "Content-Type": "application/json", "Authorization": f"Bearer {self.access_token}" } response = requests.post(url, headers=headers) return response.json() def get_order_details(self, order_id): """Get order details.""" url = f"{self.base_url}/v2/checkout/orders/{order_id}" headers = { "Authorization": f"Bearer {self.access_token}" } response = requests.get(url, headers=headers) return response.json() ``` ## IPN (Instant Payment Notification) Handling ### IPN Verification and Processing ```python from flask import Flask, request import requests from urllib.parse import parse_qs app = Flask(__name__) @app.route('/ipn', methods=['POST']) def handle_ipn(): """Handle PayPal IPN notifications.""" # Get IPN message ipn_data = request.form.to_dict() # Verify IPN with PayPal if not verify_ipn(ipn_data): return 'IPN verification failed', 400 # Process IPN based on transaction type payment_status = ipn_data.get('payment_status') txn_type = ipn_data.get('txn_type') if payment_status == 'Completed': handle_payment_completed(ipn_data) elif payment_status == 'Refunded': handle_refund(ipn_data) elif payment_status == 'Reversed': handle_chargeback(ipn_data) return 'IPN processed', 200 def verify_ipn(ipn_data): """Verify IPN message authenticity.""" # Add 'cmd' parameter verify_data = ipn_data.copy() verify_data['cmd'] = '_notify-validate' # Send back to PayPal for verification paypal_url = 'https://ipnpb.sandbox.paypal.com/cgi-bin/webscr' # or production URL response = requests.post(paypal_url, data=verify_data) return response.text == 'VERIFIED' def handle_payment_completed(ipn_data): """Process completed payment.""" txn_id = ipn_data.get('txn_id') payer_email = ipn_data.get('payer_email') mc_gross = ipn_data.get('mc_gross') item_name = ipn_data.get('item_name') # Check if already processed (prevent duplicates) if is_transaction_processed(txn_id): return # Update database # Send confirmation email # Fulfill order print(f"Payment completed: {txn_id}, Amount: ${mc_gross}") def handle_refund(ipn_data): """Handle refund.""" parent_txn_id = ipn_data.get('parent_txn_id') mc_gross = ipn_data.get('mc_gross') # Process refund in your system print(f"Refund processed: {parent_txn_id}, Amount: ${mc_gross}") def handle_chargeback(ipn_data): """Handle payment reversal/chargeback.""" txn_id = ipn_data.get('txn_id') reason_code = ipn_data.get('reason_code') # Handle chargeback print(f"Chargeback: {txn_id}, Reason: {reason_code}") ``` ## Subscription/Recurring Billing ### Create Subscription Plan ```python def create_subscription_plan(name, amount, interval='MONTH'): """Create a subscription plan.""" client = PayPalClient(CLIENT_ID, CLIENT_SECRET) url = f"{client.base_url}/v1/billing/plans" headers = { "Content-Type": "application/json", "Authorization": f"Bearer {client.access_token}" } payload = { "product_id": "PRODUCT_ID", # Create product first "name": name, "billing_cycles": [{ "frequency": { "interval_unit": interval, "interval_count": 1 }, "tenure_type": "REGULAR", "sequence": 1, "total_cycles": 0, # Infinite "pricing_scheme": { "fixed_price": { "value": str(amount), "currency_code": "USD" } } }], "payment_preferences": { "auto_bill_outstanding": True, "setup_fee": { "value": "0", "currency_code": "USD" }, "setup_fee_failure_action": "CONTINUE", "payment_failure_threshold": 3 } } response = requests.post(url, headers=headers, json=payload) return response.json() def create_subscription(plan_id, subscriber_email): """Create a subscription for a customer.""" client = PayPalClient(CLIENT_ID, CLIENT_SECRET) url = f"{client.base_url}/v1/billing/subscriptions" headers = { "Content-Type": "application/json", "Authorization": f"Bearer {client.access_token}" } payload = { "plan_id": plan_id, "subscriber": { "email_address": subscriber_email }, "application_context": { "return_url": "https://yourdomain.com/subscription/success", "cancel_url": "https://yourdomain.com/subscription/cancel" } } response = requests.post(url, headers=headers, json=payload) subscription = response.json() # Get approval URL for link in subscription.get('links', []): if link['rel'] == 'approve': return { 'subscription_id': subscription['id'], 'approval_url': link['href'] } ``` ## Refund Workflows ```python def create_refund(capture_id, amount=None, note=None): """Create a refund for a captured payment.""" client = PayPalClient(CLIENT_ID, CLIENT_SECRET) url = f"{client.base_url}/v2/payments/captures/{capture_id}/refund" headers = { "Content-Type": "application/json", "Authorization": f"Bearer {client.access_token}" } payload = {} if amount: payload["amount"] = { "value": str(amount), "currency_code": "USD" } if note: payload["note_to_payer"] = note response = requests.post(url, headers=headers, json=payload) return response.json() def get_refund_details(refund_id): """Get refund details.""" client = PayPalClient(CLIENT_ID, CLIENT_SECRET) url = f"{client.base_url}/v2/payments/refunds/{refund_id}" headers = { "Authorization": f"Bearer {client.access_token}" } response = requests.get(url, headers=headers) return response.json() ``` ## Error Handling ```python class PayPalError(Exception): """Custom PayPal error.""" pass def handle_paypal_api_call(api_function): """Wrapper for PayPal API calls with error handling.""" try: result = api_function() return result except requests.exceptions.RequestException as e: # Network error raise PayPalError(f"Network error: {str(e)}") except Exception as e: # Other errors raise PayPalError(f"PayPal API error: {str(e)}") # Usage try: order = handle_paypal_api_call(lambda: client.create_order(25.00)) except PayPalError as e: # Handle error appropriately log_error(e) ``` ## Testing ```python # Use sandbox credentials SANDBOX_CLIENT_ID = "..." SANDBOX_SECRET = "..." # Test accounts # Create test buyer and seller accounts at developer.paypal.com def test_payment_flow(): """Test complete payment flow.""" client = PayPalClient(SANDBOX_CLIENT_ID, SANDBOX_SECRET, mode='sandbox') # Create order order = client.create_order(10.00) assert 'id' in order # Get approval URL approval_url = next((link['href'] for link in order['links'] if link['rel'] == 'approve'), None) assert approval_url is not None # After approval (manual step with test account) # Capture order # captured = client.capture_order(order['id']) # assert captured['status'] == 'COMPLETED' ``` ## Resources - **references/express-checkout.md**: Express Checkout implementation guide - **references/ipn-handling.md**: IPN verification and processing - **references/refund-workflows.md**: Refund handling patterns - **references/billing-agreements.md**: Recurring billing setup - **assets/paypal-client.py**: Production PayPal client - **assets/ipn-processor.py**: IPN webhook processor - **assets/recurring-billing.py**: Subscription management ## Best Practices 1. **Always Verify IPN**: Never trust IPN without verification 2. **Idempotent Processing**: Handle duplicate IPN notifications 3. **Error Handling**: Implement robust error handling 4. **Logging**: Log all transactions and errors 5. **Test Thoroughly**: Use sandbox extensively 6. **Webhook Backup**: Don't rely solely on client-side callbacks 7. **Currency Handling**: Always specify currency explicitly ## Common Pitfalls - **Not Verifying IPN**: Accepting IPN without verification - **Duplicate Processing**: Not checking for duplicate transactions - **Wrong Environment**: Mixing sandbox and production URLs/credentials - **Missing Webhooks**: Not handling all payment states - **Hardcoded Values**: Not making configurable for different environments
👍0
👁️0
🤖 Auto-discovered
🤖system prompt•7 months ago

pci-compliance

Implement PCI DSS compliance requirements for secure handling of

business
⭐1
# PCI Compliance Master PCI DSS (Payment Card Industry Data Security Standard) compliance for secure payment processing and handling of cardholder data. ## When to Use This Skill - Building payment processing systems - Handling credit card information - Implementing secure payment flows - Conducting PCI compliance audits - Reducing PCI compliance scope - Implementing tokenization and encryption - Preparing for PCI DSS assessments ## PCI DSS Requirements (12 Core Requirements) ### Build and Maintain Secure Network 1. Install and maintain firewall configuration 2. Don't use vendor-supplied defaults for passwords ### Protect Cardholder Data 3. Protect stored cardholder data 4. Encrypt transmission of cardholder data across public networks ### Maintain Vulnerability Management 5. Protect systems against malware 6. Develop and maintain secure systems and applications ### Implement Strong Access Control 7. Restrict access to cardholder data by business need-to-know 8. Identify and authenticate access to system components 9. Restrict physical access to cardholder data ### Monitor and Test Networks 10. Track and monitor all access to network resources and cardholder data 11. Regularly test security systems and processes ### Maintain Information Security Policy 12. Maintain a policy that addresses information security ## Compliance Levels **Level 1**: > 6 million transactions/year (annual ROC required) **Level 2**: 1-6 million transactions/year (annual SAQ) **Level 3**: 20,000-1 million e-commerce transactions/year **Level 4**: < 20,000 e-commerce or < 1 million total transactions ## Data Minimization (Never Store) ```python # NEVER STORE THESE PROHIBITED_DATA = { 'full_track_data': 'Magnetic stripe data', 'cvv': 'Card verification code/value', 'pin': 'PIN or PIN block' } # CAN STORE (if encrypted) ALLOWED_DATA = { 'pan': 'Primary Account Number (card number)', 'cardholder_name': 'Name on card', 'expiration_date': 'Card expiration', 'service_code': 'Service code' } class PaymentData: """Safe payment data handling.""" def __init__(self): self.prohibited_fields = ['cvv', 'cvv2', 'cvc', 'pin'] def sanitize_log(self, data): """Remove sensitive data from logs.""" sanitized = data.copy() # Mask PAN if 'card_number' in sanitized: card = sanitized['card_number'] sanitized['card_number'] = f"{card[:6]}{'*' * (len(card) - 10)}{card[-4:]}" # Remove prohibited data for field in self.prohibited_fields: sanitized.pop(field, None) return sanitized def validate_no_prohibited_storage(self, data): """Ensure no prohibited data is being stored.""" for field in self.prohibited_fields: if field in data: raise SecurityError(f"Attempting to store prohibited field: {field}") ``` ## Tokenization ### Using Payment Processor Tokens ```python import stripe class TokenizedPayment: """Handle payments using tokens (no card data on server).""" @staticmethod def create_payment_method_token(card_details): """Create token from card details (client-side only).""" # THIS SHOULD ONLY BE DONE CLIENT-SIDE WITH STRIPE.JS # NEVER send card details to your server """ // Frontend JavaScript const stripe = Stripe('pk_...'); const {token, error} = await stripe.createToken({ card: { number: '4242424242424242', exp_month: 12, exp_year: 2024, cvc: '123' } }); // Send token.id to server (NOT card details) """ pass @staticmethod def charge_with_token(token_id, amount): """Charge using token (server-side).""" # Your server only sees the token, never the card number stripe.api_key = "sk_..." charge = stripe.Charge.create( amount=amount, currency="usd", source=token_id, # Token instead of card details description="Payment" ) return charge @staticmethod def store_payment_method(customer_id, payment_method_token): """Store payment method as token for future use.""" stripe.Customer.modify( customer_id, source=payment_method_token ) # Store only customer_id and payment_method_id in your database # NEVER store actual card details return { 'customer_id': customer_id, 'has_payment_method': True # DO NOT store: card number, CVV, etc. } ``` ### Custom Tokenization (Advanced) ```python import secrets from cryptography.fernet import Fernet class TokenVault: """Secure token vault for card data (if you must store it).""" def __init__(self, encryption_key): self.cipher = Fernet(encryption_key) self.vault = {} # In production: use encrypted database def tokenize(self, card_data): """Convert card data to token.""" # Generate secure random token token = secrets.token_urlsafe(32) # Encrypt card data encrypted = self.cipher.encrypt(json.dumps(card_data).encode()) # Store token -> encrypted data mapping self.vault[token] = encrypted return token def detokenize(self, token): """Retrieve card data from token.""" encrypted = self.vault.get(token) if not encrypted: raise ValueError("Token not found") # Decrypt decrypted = self.cipher.decrypt(encrypted) return json.loads(decrypted.decode()) def delete_token(self, token): """Remove token from vault.""" self.vault.pop(token, None) ``` ## Encryption ### Data at Rest ```python from cryptography.hazmat.primitives.ciphers.aead import AESGCM import os class EncryptedStorage: """Encrypt data at rest using AES-256-GCM.""" def __init__(self, encryption_key): """Initialize with 256-bit key.""" self.key = encryption_key # Must be 32 bytes def encrypt(self, plaintext): """Encrypt data.""" # Generate random nonce nonce = os.urandom(12) # Encrypt aesgcm = AESGCM(self.key) ciphertext = aesgcm.encrypt(nonce, plaintext.encode(), None) # Return nonce + ciphertext return nonce + ciphertext def decrypt(self, encrypted_data): """Decrypt data.""" # Extract nonce and ciphertext nonce = encrypted_data[:12] ciphertext = encrypted_data[12:] # Decrypt aesgcm = AESGCM(self.key) plaintext = aesgcm.decrypt(nonce, ciphertext, None) return plaintext.decode() # Usage storage = EncryptedStorage(os.urandom(32)) encrypted_pan = storage.encrypt("4242424242424242") # Store encrypted_pan in database ``` ### Data in Transit ```python # Always use TLS 1.2 or higher # Flask/Django example app.config['SESSION_COOKIE_SECURE'] = True # HTTPS only app.config['SESSION_COOKIE_HTTPONLY'] = True app.config['SESSION_COOKIE_SAMESITE'] = 'Strict' # Enforce HTTPS from flask_talisman import Talisman Talisman(app, force_https=True) ``` ## Access Control ```python from functools import wraps from flask import session def require_pci_access(f): """Decorator to restrict access to cardholder data.""" @wraps(f) def decorated_function(*args, **kwargs): user = session.get('user') # Check if user has PCI access role if not user or 'pci_access' not in user.get('roles', []): return {'error': 'Unauthorized access to cardholder data'}, 403 # Log access attempt audit_log( user=user['id'], action='access_cardholder_data', resource=f.__name__ ) return f(*args, **kwargs) return decorated_function @app.route('/api/payment-methods') @require_pci_access def get_payment_methods(): """Retrieve payment methods (restricted access).""" # Only accessible to users with pci_access role pass ``` ## Audit Logging ```python import logging from datetime import datetime class PCIAuditLogger: """PCI-compliant audit logging.""" def __init__(self): self.logger = logging.getLogger('pci_audit') # Configure to write to secure, append-only log def log_access(self, user_id, resource, action, result): """Log access to cardholder data.""" entry = { 'timestamp': datetime.utcnow().isoformat(), 'user_id': user_id, 'resource': resource, 'action': action, 'result': result, 'ip_address': request.remote_addr } self.logger.info(json.dumps(entry)) def log_authentication(self, user_id, success, method): """Log authentication attempt.""" entry = { 'timestamp': datetime.utcnow().isoformat(), 'user_id': user_id, 'event': 'authentication', 'success': success, 'method': method, 'ip_address': request.remote_addr } self.logger.info(json.dumps(entry)) # Usage audit = PCIAuditLogger() audit.log_access(user_id=123, resource='payment_methods', action='read', result='success') ``` ## Security Best Practices ### Input Validation ```python import re def validate_card_number(card_number): """Validate card number format (Luhn algorithm).""" # Remove spaces and dashes card_number = re.sub(r'[\s-]', '', card_number) # Check if all digits if not card_number.isdigit(): return False # Luhn algorithm def luhn_checksum(card_num): def digits_of(n): return [int(d) for d in str(n)] digits = digits_of(card_num) odd_digits = digits[-1::-2] even_digits = digits[-2::-2] checksum = sum(odd_digits) for d in even_digits: checksum += sum(digits_of(d * 2)) return checksum % 10 return luhn_checksum(card_number) == 0 def sanitize_input(user_input): """Sanitize user input to prevent injection.""" # Remove special characters # Validate against expected format # Escape for database queries pass ``` ## PCI DSS SAQ (Self-Assessment Questionnaire) ### SAQ A (Least Requirements) - E-commerce using hosted payment page - No card data on your systems - ~20 questions ### SAQ A-EP - E-commerce with embedded payment form - Uses JavaScript to handle card data - ~180 questions ### SAQ D (Most Requirements) - Store, process, or transmit card data - Full PCI DSS requirements - ~300 questions ## Compliance Checklist ```python PCI_COMPLIANCE_CHECKLIST = { 'network_security': [ 'Firewall configured and maintained', 'No vendor default passwords', 'Network segmentation implemented' ], 'data_protection': [ 'No storage of CVV, track data, or PIN', 'PAN encrypted when stored', 'PAN masked when displayed', 'Encryption keys properly managed' ], 'vulnerability_management': [ 'Anti-virus installed and updated', 'Secure development practices', 'Regular security patches', 'Vulnerability scanning performed' ], 'access_control': [ 'Access restricted by role', 'Unique IDs for all users', 'Multi-factor authentication', 'Physical security measures' ], 'monitoring': [ 'Audit logs enabled', 'Log review process', 'File integrity monitoring', 'Regular security testing' ], 'policy': [ 'Security policy documented', 'Risk assessment performed', 'Security awareness training', 'Incident response plan' ] } ``` ## Resources - **references/data-minimization.md**: Never store prohibited data - **references/tokenization.md**: Tokenization strategies - **references/encryption.md**: Encryption requirements - **references/access-control.md**: Role-based access - **references/audit-logging.md**: Comprehensive logging - **assets/pci-compliance-checklist.md**: Complete checklist - **assets/encrypted-storage.py**: Encryption utilities - **scripts/audit-payment-system.sh**: Compliance audit script ## Common Violations 1. **Storing CVV**: Never store card verification codes 2. **Unencrypted PAN**: Card numbers must be encrypted at rest 3. **Weak Encryption**: Use AES-256 or equivalent 4. **No Access Controls**: Restrict who can access cardholder data 5. **Missing Audit Logs**: Must log all access to payment data 6. **Insecure Transmission**: Always use TLS 1.2+ 7. **Default Passwords**: Change all default credentials 8. **No Security Testing**: Regular penetration testing required ## Reducing PCI Scope 1. **Use Hosted Payments**: Stripe Checkout, PayPal, etc. 2. **Tokenization**: Replace card data with tokens 3. **Network Segmentation**: Isolate cardholder data environment 4. **Outsource**: Use PCI-compliant payment processors 5. **No Storage**: Never store full card details By minimizing systems that touch card data, you reduce compliance burden significantly.
👍0
👁️0
🤖 Auto-discovered
🤖system prompt•7 months ago

stripe-integration

Implement Stripe payment processing for robust, PCI-compliant

business
⭐1
# Stripe Integration Master Stripe payment processing integration for robust, PCI-compliant payment flows including checkout, subscriptions, webhooks, and refunds. ## When to Use This Skill - Implementing payment processing in web/mobile applications - Setting up subscription billing systems - Handling one-time payments and recurring charges - Processing refunds and disputes - Managing customer payment methods - Implementing SCA (Strong Customer Authentication) for European payments - Building marketplace payment flows with Stripe Connect ## Core Concepts ### 1. Payment Flows **Checkout Sessions** - Recommended for most integrations - Supports all UI paths: - Stripe-hosted checkout page - Embedded checkout form - Custom UI with Elements (Payment Element, Express Checkout Element) using `ui_mode='custom'` - Provides built-in checkout capabilities (line items, discounts, tax, shipping, address collection, saved payment methods, and checkout lifecycle events) - Lower integration and maintenance burden than Payment Intents **Payment Intents (Bespoke control)** - You calculate the final amount with taxes, discounts, subscriptions, and currency conversion yourself. - More complex implementation and long-term maintenance burden - Requires Stripe.js for PCI compliance **Setup Intents (Save Payment Methods)** - Collect payment method without charging - Used for subscriptions and future payments - Requires customer confirmation ### 2. Webhooks **Critical Events:** - `payment_intent.succeeded`: Payment completed - `payment_intent.payment_failed`: Payment failed - `customer.subscription.updated`: Subscription changed - `customer.subscription.deleted`: Subscription canceled - `charge.refunded`: Refund processed - `invoice.payment_succeeded`: Subscription payment successful ### 3. Subscriptions **Components:** - **Product**: What you're selling - **Price**: How much and how often - **Subscription**: Customer's recurring payment - **Invoice**: Generated for each billing cycle ### 4. Customer Management - Create and manage customer records - Store multiple payment methods - Track customer metadata - Manage billing details ## Quick Start ```python import stripe stripe.api_key = "sk_test_..." # Create a checkout session session = stripe.checkout.Session.create( line_items=[{ 'price_data': { 'currency': 'usd', 'product_data': { 'name': 'Premium Subscription', }, 'unit_amount': 2000, # $20.00 'recurring': { 'interval': 'month', }, }, 'quantity': 1, }], mode='subscription', success_url='https://yourdomain.com/success?session_id={CHECKOUT_SESSION_ID}', cancel_url='https://yourdomain.com/cancel' ) # Redirect user to session.url print(session.url) ``` ## Payment Implementation Patterns ### Pattern 1: One-Time Payment (Hosted Checkout) ```python def create_checkout_session(amount, currency='usd'): """Create a one-time payment checkout session.""" try: session = stripe.checkout.Session.create( line_items=[{ 'price_data': { 'currency': currency, 'product_data': { 'name': 'Blue T-shirt', 'images': ['https://example.com/product.jpg'], }, 'unit_amount': amount, # Amount in cents }, 'quantity': 1, }], mode='payment', success_url='https://yourdomain.com/success?session_id={CHECKOUT_SESSION_ID}', cancel_url='https://yourdomain.com/cancel', metadata={ 'order_id': 'order_123', 'user_id': 'user_456' } ) return session except stripe.error.StripeError as e: # Handle error print(f"Stripe error: {e.user_message}") raise ``` ### Pattern 2: Elements with Checkout Sessions ```python def create_checkout_session_for_elements(amount, currency='usd'): """Create a checkout session configured for Payment Element.""" session = stripe.checkout.Session.create( mode='payment', ui_mode='custom', line_items=[{ 'price_data': { 'currency': currency, 'product_data': {'name': 'Blue T-shirt'}, 'unit_amount': amount, }, 'quantity': 1, }], return_url='https://yourdomain.com/complete?session_id={CHECKOUT_SESSION_ID}' ) return session.client_secret # Send to frontend ``` ```javascript const stripe = Stripe("pk_test_..."); const appearance = { theme: "stripe" }; const checkout = stripe.initCheckout({ clientSecret, elementsOptions: { appearance }, }); const loadActionsResult = await checkout.loadActions(); if (loadActionsResult.type === "success") { const { actions } = loadActionsResult; const session = actions.getSession(); const button = document.getElementById("pay-button"); const checkoutContainer = document.getElementById("checkout-container"); const emailInput = document.getElementById("email"); const emailErrors = document.getElementById("email-errors"); const errors = document.getElementById("confirm-errors"); // Display a formatted string representing the total amount checkoutContainer.append(`Total: ${session.total.total.amount}`); // Mount Payment Element const paymentElement = checkout.createPaymentElement(); paymentElement.mount("#payment-element"); // Store email for submission emailInput.addEventListener("blur", () => { actions.updateEmail(emailInput.value).then((result) => { if (result.error) emailErrors.textContent = result.error.message; }); }); // Handle form submission button.addEventListener("click", () => { actions.confirm().then((result) => { if (result.type === "error") errors.textContent = result.error.message; }); }); } ``` ### Pattern 3: Elements with Payment Intents Pattern 2 (Elements with Checkout Sessions) is Stripe's recommended approach, but you can also use Payment Intents as an alternative. ```python def create_payment_intent(amount, currency='usd', customer_id=None): """Create a payment intent for bespoke checkout UI with Payment Element.""" intent = stripe.PaymentIntent.create( amount=amount, currency=currency, customer=customer_id, automatic_payment_methods={ 'enabled': True, }, metadata={ 'integration_check': 'accept_a_payment' } ) return intent.client_secret # Send to frontend ``` ```javascript // Mount Payment Element and confirm via Payment Intents const stripe = Stripe("pk_test_..."); const appearance = { theme: "stripe" }; const elements = stripe.elements({ appearance, clientSecret }); const paymentElement = elements.create("payment"); paymentElement.mount("#payment-element"); document.getElementById("pay-button").addEventListener("click", async () => { const { error } = await stripe.confirmPayment({ elements, confirmParams: { return_url: "https://yourdomain.com/complete", }, }); if (error) { document.getElementById("errors").textContent = error.message; } }); ``` ### Pattern 4: Subscription Creation ```python def create_subscription(customer_id, price_id): """Create a subscription for a customer.""" try: subscription = stripe.Subscription.create( customer=customer_id, items=[{'price': price_id}], payment_behavior='default_incomplete', payment_settings={'save_default_payment_method': 'on_subscription'}, expand=['latest_invoice.payment_intent'], ) return { 'subscription_id': subscription.id, 'client_secret': subscription.latest_invoice.payment_intent.client_secret } except stripe.error.StripeError as e: print(f"Subscription creation failed: {e}") raise ``` ### Pattern 5: Customer Portal ```python def create_customer_portal_session(customer_id): """Create a portal session for customers to manage subscriptions.""" session = stripe.billing_portal.Session.create( customer=customer_id, return_url='https://yourdomain.com/account', ) return session.url # Redirect customer here ``` ## Webhook Handling ### Secure Webhook Endpoint ```python from flask import Flask, request import stripe app = Flask(__name__) endpoint_secret = 'whsec_...' @app.route('/webhook', methods=['POST']) def webhook(): payload = request.data sig_header = request.headers.get('Stripe-Signature') try: event = stripe.Webhook.construct_event( payload, sig_header, endpoint_secret ) except ValueError: # Invalid payload return 'Invalid payload', 400 except stripe.error.SignatureVerificationError: # Invalid signature return 'Invalid signature', 400 # Handle the event if event['type'] == 'payment_intent.succeeded': payment_intent = event['data']['object'] handle_successful_payment(payment_intent) elif event['type'] == 'payment_intent.payment_failed': payment_intent = event['data']['object'] handle_failed_payment(payment_intent) elif event['type'] == 'customer.subscription.deleted': subscription = event['data']['object'] handle_subscription_canceled(subscription) return 'Success', 200 def handle_successful_payment(payment_intent): """Process successful payment.""" customer_id = payment_intent.get('customer') amount = payment_intent['amount'] metadata = payment_intent.get('metadata', {}) # Update your database # Send confirmation email # Fulfill order print(f"Payment succeeded: {payment_intent['id']}") def handle_failed_payment(payment_intent): """Handle failed payment.""" error = payment_intent.get('last_payment_error', {}) print(f"Payment failed: {error.get('message')}") # Notify customer # Update order status def handle_subscription_canceled(subscription): """Handle subscription cancellation.""" customer_id = subscription['customer'] # Update user access # Send cancellation email print(f"Subscription canceled: {subscription['id']}") ``` ### Webhook Best Practices ```python import hashlib import hmac def verify_webhook_signature(payload, signature, secret): """Manually verify webhook signature.""" expected_sig = hmac.new( secret.encode('utf-8'), payload, hashlib.sha256 ).hexdigest() return hmac.compare_digest(signature, expected_sig) def handle_webhook_idempotently(event_id, handler): """Ensure webhook is processed exactly once.""" # Check if event already processed if is_event_processed(event_id): return # Process event try: handler() mark_event_processed(event_id) except Exception as e: log_error(e) # Stripe will retry failed webhooks raise ``` ## Customer Management ```python def create_customer(email, name, payment_method_id=None): """Create a Stripe customer.""" customer = stripe.Customer.create( email=email, name=name, payment_method=payment_method_id, invoice_settings={ 'default_payment_method': payment_method_id } if payment_method_id else None, metadata={ 'user_id': '12345' } ) return customer def attach_payment_method(customer_id, payment_method_id): """Attach a payment method to a customer.""" stripe.PaymentMethod.attach( payment_method_id, customer=customer_id ) # Set as default stripe.Customer.modify( customer_id, invoice_settings={ 'default_payment_method': payment_method_id } ) def list_customer_payment_methods(customer_id): """List all payment methods for a customer.""" payment_methods = stripe.PaymentMethod.list( customer=customer_id, type='card' ) return payment_methods.data ``` ## Refund Handling ```python def create_refund(payment_intent_id, amount=None, reason=None): """Create a refund.""" refund_params = { 'payment_intent': payment_intent_id } if amount: refund_params['amount'] = amount # Partial refund if reason: refund_params['reason'] = reason # 'duplicate', 'fraudulent', 'requested_by_customer' refund = stripe.Refund.create(**refund_params) return refund def handle_dispute(charge_id, evidence): """Update dispute with evidence.""" stripe.Dispute.modify( charge_id, evidence={ 'customer_name': evidence.get('customer_name'), 'customer_email_address': evidence.get('customer_email'), 'shipping_documentation': evidence.get('shipping_proof'), 'customer_communication': evidence.get('communication'), } ) ``` ## Testing ```python # Use test mode keys stripe.api_key = "sk_test_..." # Test card numbers TEST_CARDS = { 'success': '4242424242424242', 'declined': '4000000000000002', '3d_secure': '4000002500003155', 'insufficient_funds': '4000000000009995' } def test_payment_flow(): """Test complete payment flow.""" # Create test customer customer = stripe.Customer.create( email="test@example.com" ) # Create payment intent intent = stripe.PaymentIntent.create( amount=1000, automatic_payment_methods={ 'enabled': True }, currency='usd', customer=customer.id ) # Confirm with test card confirmed = stripe.PaymentIntent.confirm( intent.id, payment_method='pm_card_visa' # Test payment method ) assert confirmed.status == 'succeeded' ``` ## Resources - **references/checkout-flows.md**: Detailed checkout implementation - **references/webhook-handling.md**: Webhook security and processing - **references/subscription-management.md**: Subscription lifecycle - **references/customer-management.md**: Customer and payment method handling - **references/invoice-generation.md**: Invoicing and billing - **assets/stripe-client.py**: Production-ready Stripe client wrapper - **assets/webhook-handler.py**: Complete webhook processor - **assets/checkout-config.json**: Checkout configuration templates ## Best Practices 1. **Always Use Webhooks**: Don't rely solely on client-side confirmation 2. **Idempotency**: Handle webhook events idempotently 3. **Error Handling**: Gracefully handle all Stripe errors 4. **Test Mode**: Thoroughly test with test keys before production 5. **Metadata**: Use metadata to link Stripe objects to your database 6. **Monitoring**: Track payment success rates and errors 7. **PCI Compliance**: Never handle raw card data on your server 8. **SCA Ready**: Implement 3D Secure for European payments ## Common Pitfalls - **Not Verifying Webhooks**: Always verify webhook signatures - **Missing Webhook Events**: Handle all relevant webhook events - **Hardcoded Amounts**: Use cents/smallest currency unit - **No Retry Logic**: Implement retries for API calls - **Ignoring Test Mode**: Test all edge cases with test cards
👍0
👁️0
🤖 Auto-discovered