Skip to content
← Back to snippets
PythonTypeScript

Python and TypeScript utilities: async, retries, typing

Ten snippets across two languages: bounded asyncio.gather, a retry decorator with backoff, CSV reading and list flattening in Python; array chunking, grouping, offset pagination, an abortable fetch and Zod validation schemas in TypeScript.

Different languages, same three problems: doing many things at once without doing all of them at once, retrying what failed, and being sure the data is the shape you think it is.

Unbounded concurrency is a denial of service you wrote yourself

asyncio.gather(*tasks) starts every task immediately. With ten tasks that is fine; with ten thousand URLs it opens ten thousand connections, and the thing that falls over is usually the service you are calling rather than yours. A Semaphore wrapping each task caps the number in flight while keeping the same interface, and choosing that cap is a real decision — it is the rate you are willing to impose on someone else's infrastructure.

Retries need a ceiling and a random component

A retry loop without exponential backoff hammers a struggling service at full rate and makes recovery slower. Backoff alone is not enough either: if a hundred clients fail at the same instant, they all back off by the same amount and retry in the same instant, reproducing the spike. Jitter — randomising the delay — is what spreads them out, and it is the part most often left out.

The other half of the decision is what to retry. A 500 or a timeout is worth retrying; a 400 or a 403 will fail identically every time, and retrying it just delays the error you were always going to get.

Chunking, grouping and the pagination trade-off

The TypeScript utilities are small by design. Chunking an array matters mainly because APIs have batch limits and databases have parameter limits, so the size is dictated externally rather than chosen. Grouping by key is Object.groupBy in current runtimes, and the snippet shows the typed hand-rolled version for environments that do not have it yet.

Offset pagination is included with its cost stated: the database still scans the rows it skips, so page 1000 is slower than page 1. It is the right choice when the UI shows page numbers, and the wrong one for an infinite scroll — that is what cursors are for.

Validation at the boundary

Zod's value is not that it validates, it is that the validated result carries a static type derived from the same schema — so the runtime check and the compile-time type cannot drift apart. Declaring an interface and casting an API response to it asserts a shape rather than checking one, and the assertion is wrong the moment the API changes. Parsing at the boundary means a malformed response fails where it arrived instead of three layers deeper.

PythonPython

asyncio.gather with Concurrency Limit

Runs a list of async tasks in parallel, but no more than N at a time — using asyncio.Semaphore. Prevents overloading external APIs or databases when processing large task lists.

How it works: asyncio.Semaphore(limit) allows at most limit tasks to run simultaneously. Each coroutine is wrapped in bounded, which waits for a free slot via async with semaphore. asyncio.gather launches everything in parallel, the semaphore controls the flow.

import asyncio
from typing import TypeVar, Coroutine, Any

T = TypeVar('T')


async def gather_limited(coros: list[Coroutine[Any, Any, T]], limit: int = 10) -> list[T]:
    """Run coroutines in parallel, at most `limit` at a time."""
    semaphore = asyncio.Semaphore(limit)

    async def bounded(coro: Coroutine[Any, Any, T]) -> T:
        async with semaphore:
            return await coro

    return await asyncio.gather(*(bounded(c) for c in coros))


# Example usage — fetch 100 URLs, at most 5 in parallel
import aiohttp

async def fetch(session: aiohttp.ClientSession, url: str) -> dict:
    async with session.get(url) as resp:
        return await resp.json()

async def main():
    urls = [f'https://api.example.com/items/{i}' for i in range(100)]
    async with aiohttp.ClientSession() as session:
        results = await gather_limited(
            [fetch(session, url) for url in urls],
            limit=5,
        )
    return results
PythonPython

Check if a file or directory exists

The modern, cross-platform way to check for a file's existence using pathlib instead of the older os.path functions.

How it works: pathlib.Path wraps filesystem paths in an object with .is_file()/.is_dir() methods, which is more readable and less error-prone than string-based os.path.isfile()/os.path.isdir() calls.

from pathlib import Path

def file_exists(path):
    return Path(path).is_file()

def dir_exists(path):
    return Path(path).is_dir()

# Example usage
if file_exists('config.json'):
    print('Config found')
PythonPython

Flatten a nested list

Recursively flatten an arbitrarily nested list into a single flat list.

How it works: for each item, if it's itself a list the function recurses into it and extends the result; otherwise the item is appended as-is.

def flatten(items):
    result = []
    for item in items:
        if isinstance(item, list):
            result.extend(flatten(item))
        else:
            result.append(item)
    return result

# Example usage
flatten([1, [2, 3, [4, 5]], 6])  # -> [1, 2, 3, 4, 5, 6]
PythonPython

Read a CSV file into a list of dictionaries

The standard way to parse a CSV file's header row into keys, using Python's built-in csv module — no pandas needed for simple cases.

How it works: csv.DictReader uses the first row as field names automatically, so each row comes back as a dict you can access by column name instead of a fragile numeric index.

import csv

def read_csv(path):
    with open(path, newline='', encoding='utf-8') as f:
        return list(csv.DictReader(f))

# Example usage
rows = read_csv('users.csv')
for row in rows:
    print(row['name'], row['email'])
PythonPython

Retry Decorator with Backoff (Python)

A decorator for retrying unstable functions with exponential backoff and jitter. Configurable exceptions, attempt count, and delay.

How it works: @wraps(func) preserves the original function's metadata. On each failure the delay doubles (base_delay * 2 ** attempt), capped at max_delay. A 0–20% jitter spreads retries from many clients across time.

import time
import random
from functools import wraps


def retry(max_attempts=3, base_delay=0.2, max_delay=2.0, exceptions=(Exception,)):
    """Retry a call with exponential backoff and jitter."""
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            for attempt in range(1, max_attempts + 1):
                try:
                    return func(*args, **kwargs)
                except exceptions as exc:
                    if attempt >= max_attempts:
                        raise
                    delay = min(max_delay, base_delay * 2 ** (attempt - 1))
                    jitter = random.uniform(0, delay * 0.2)
                    time.sleep(delay + jitter)
        return wrapper
    return decorator


# Example usage
import requests

@retry(max_attempts=4, base_delay=0.5, exceptions=(requests.RequestException,))
def fetch_data(url: str) -> dict:
    response = requests.get(url, timeout=5)
    response.raise_for_status()
    return response.json()


data = fetch_data('https://api.example.com/items')
TypeScriptTypeScript

Chunk Array into Groups

Splits an array into groups of a given size — useful for batch processing, in-memory pagination, or sending data in parts.

How it works: iterates over the array in steps of size, slicing a chunk at each step. The last chunk may be smaller than size if there aren't enough elements.

function chunk<T>(array: T[], size: number): T[][] {
  if (size <= 0) throw new RangeError('chunk size must be > 0')
  const chunks: T[][] = []
  for (let i = 0; i < array.length; i += size) {
    chunks.push(array.slice(i, i + size))
  }
  return chunks
}

// Example usage
chunk([1, 2, 3, 4, 5], 2)  // [[1,2],[3,4],[5]]
chunk(['a','b','c','d'], 3) // [['a','b','c'],['d']]

// Batch Telegram sends (30 msg/sec limit)
const userBatches = chunk(userIds, 25)
for (const batch of userBatches) {
  await Promise.all(batch.map((id) => sendMessage(id, text)))
  await sleep(1000)
}
TypeScriptTypeScript

Group Array by Key

A lodash-free groupBy — groups an array of objects by a field value into a Record. TypeScript-typed, with a note on the native Object.groupBy alternative.

How it works: uses reduce to accumulate items into an object where keys are the grouping field values and values are arrays of matching elements. In modern environments (Node 21+) you can use the native Object.groupBy instead.

function groupBy<T>(array: T[], key: keyof T): Record<string, T[]> {
  return array.reduce((groups, item) => {
    const groupKey = String(item[key])
    groups[groupKey] = [...(groups[groupKey] ?? []), item]
    return groups
  }, {} as Record<string, T[]>)
}

// Native alternative (Node 21+ / Chrome 117+)
// const grouped = Object.groupBy(array, (item) => item.status)

// Example usage
const orders = [
  { id: 1, status: 'paid' },
  { id: 2, status: 'pending' },
  { id: 3, status: 'paid' },
]
groupBy(orders, 'status')
// { paid: [{id:1,...},{id:3,...}], pending: [{id:2,...}] }
TypeScriptTypeScript

Pagination Utilities (Offset / Limit)

Typed helpers for server-side pagination — compute offset and limit from a page number, return metadata (totalPages, hasNext, hasPrev).

How it works: paginate converts page/perPage into offset/limit for a SQL query. paginationMeta takes the total count and returns all metadata for a consistent API response format.

interface PaginationInput {
  page: number
  perPage?: number
}

interface PaginationMeta {
  total: number
  page: number
  perPage: number
  totalPages: number
  hasNext: boolean
  hasPrev: boolean
}

function paginate({ page, perPage = 20 }: PaginationInput) {
  const safePage = Math.max(1, Math.floor(page))
  const limit = Math.max(1, Math.min(perPage, 100))
  const offset = (safePage - 1) * limit
  return { limit, offset, page: safePage }
}

function paginationMeta(total: number, input: PaginationInput): PaginationMeta {
  const { page } = paginate(input)
  const perPage = input.perPage ?? 20
  const totalPages = Math.ceil(total / perPage) || 1
  return { total, page, perPage, totalPages, hasNext: page < totalPages, hasPrev: page > 1 }
}

// Example usage — SQL query
const { limit, offset } = paginate({ page: 3, perPage: 10 })
// SELECT * FROM orders LIMIT 10 OFFSET 20

const meta = paginationMeta(135, { page: 3, perPage: 10 })
// { total: 135, page: 3, perPage: 10, totalPages: 14, hasNext: true, hasPrev: true }
TypeScriptTypeScript

useFetch with AbortController — TypeScript

A Vue composable for fetch requests that automatically cancels the previous request when a new one is made. Reactive data, loading, and error states included.

How it works: every new execute call first cancels the previous request via AbortController.abort(). This prevents race conditions — a stale response won't overwrite a fresh one. AbortError is ignored; all other errors are written to error.

import { ref } from 'vue'

export function useFetch<T>() {
  const data = ref<T | null>(null)
  const loading = ref(false)
  const error = ref<Error | null>(null)

  let controller: AbortController | null = null

  async function execute(url: string, options: RequestInit = {}) {
    // Cancel the previous request
    controller?.abort()
    controller = new AbortController()

    loading.value = true
    error.value = null

    try {
      const res = await fetch(url, { ...options, signal: controller.signal })
      if (!res.ok) throw new Error(`HTTP ${res.status}: ${res.statusText}`)
      data.value = await res.json()
    } catch (e) {
      if ((e as Error).name !== 'AbortError') {
        error.value = e as Error
      }
    } finally {
      loading.value = false
    }
  }

  function abort() {
    controller?.abort()
  }

  return { data, loading, error, execute, abort }
}

// Example usage
const { data: users, loading, error, execute } = useFetch<User[]>()

// Switching pages — the old request is cancelled automatically
watch(currentPage, (page) => {
  execute(`/api/users?page=${page}`)
})
TypeScriptTypeScript

Zod API validation schemas

Define type-safe request validation with Zod — parse, transform, and validate API request bodies, query strings, and URL params.

How it works: Zod schemas define the expected shape and constraints of incoming data. parse throws on invalid data; safeParse returns a result object so you can handle errors gracefully without try/catch.

import { z } from 'zod'

const CreateUserSchema = z.object({
  email: z.string().email(),
  name: z.string().min(2).max(100),
  age: z.number().int().positive().optional(),
  role: z.enum(['user', 'admin']).default('user'),
})

// Use with safeParse for readable errors
const result = CreateUserSchema.safeParse(body)

if (!result.success) {
  const messages = result.error.issues.map(
    issue => `${issue.path.join('.')}: ${issue.message}`
  )
  return { ok: false, errors: messages }
}

// result.data is fully typed
createUser(result.data)
// Reusable fragments
const PaginationSchema = z.object({
  page: z.coerce.number().int().positive().default(1),
  limit: z.coerce.number().int().min(1).max(100).default(20),
})

const UuidParam = z.object({
  id: z.string().uuid(),
})

// Express handler example
app.post('/api/users', (req, res) => {
  const parsed = CreateUserSchema.safeParse(req.body)
  if (!parsed.success) return res.status(400).json(parsed.error.flatten())
  res.json(createUser(parsed.data))
})
// CONNECT

Contact Information

Submit a request or write to me directly on Telegram. I'll reply within 24 hours.

Provide either your email or Telegram so I can reply.