Python and TypeScript utilities: async, retries, typing
Ten snippets across two languages: bounded asyncio.gather, a retry decorator with backoff, CSV reading and list flattening in Python; array chunking, grouping, offset pagination, an abortable fetch and Zod validation schemas in TypeScript.
Different languages, same three problems: doing many things at once without doing all of them at once, retrying what failed, and being sure the data is the shape you think it is.
Unbounded concurrency is a denial of service you wrote yourself
asyncio.gather(*tasks) starts every task immediately. With ten tasks that is fine; with ten thousand URLs it opens ten thousand connections, and the thing that falls over is usually the service you are calling rather than yours. A Semaphore wrapping each task caps the number in flight while keeping the same interface, and choosing that cap is a real decision — it is the rate you are willing to impose on someone else's infrastructure.
Retries need a ceiling and a random component
A retry loop without exponential backoff hammers a struggling service at full rate and makes recovery slower. Backoff alone is not enough either: if a hundred clients fail at the same instant, they all back off by the same amount and retry in the same instant, reproducing the spike. Jitter — randomising the delay — is what spreads them out, and it is the part most often left out.
The other half of the decision is what to retry. A 500 or a timeout is worth retrying; a 400 or a 403 will fail identically every time, and retrying it just delays the error you were always going to get.
Chunking, grouping and the pagination trade-off
The TypeScript utilities are small by design. Chunking an array matters mainly because APIs have batch limits and databases have parameter limits, so the size is dictated externally rather than chosen. Grouping by key is Object.groupBy in current runtimes, and the snippet shows the typed hand-rolled version for environments that do not have it yet.
Offset pagination is included with its cost stated: the database still scans the rows it skips, so page 1000 is slower than page 1. It is the right choice when the UI shows page numbers, and the wrong one for an infinite scroll — that is what cursors are for.
Validation at the boundary
Zod's value is not that it validates, it is that the validated result carries a static type derived from the same schema — so the runtime check and the compile-time type cannot drift apart. Declaring an interface and casting an API response to it asserts a shape rather than checking one, and the assertion is wrong the moment the API changes. Parsing at the boundary means a malformed response fails where it arrived instead of three layers deeper.
asyncio.gather with Concurrency Limit
Runs a list of async tasks in parallel, but no more than N at a time — using asyncio.Semaphore. Prevents overloading external APIs or databases when processing large task lists.
How it works: asyncio.Semaphore(limit) allows at most limit tasks to run simultaneously. Each coroutine is wrapped in bounded, which waits for a free slot via async with semaphore. asyncio.gather launches everything in parallel, the semaphore controls the flow.
import asyncio
from typing import TypeVar, Coroutine, Any
T = TypeVar('T')
async def gather_limited(coros: list[Coroutine[Any, Any, T]], limit: int = 10) -> list[T]:
"""Run coroutines in parallel, at most `limit` at a time."""
semaphore = asyncio.Semaphore(limit)
async def bounded(coro: Coroutine[Any, Any, T]) -> T:
async with semaphore:
return await coro
return await asyncio.gather(*(bounded(c) for c in coros))
# Example usage — fetch 100 URLs, at most 5 in parallel
import aiohttp
async def fetch(session: aiohttp.ClientSession, url: str) -> dict:
async with session.get(url) as resp:
return await resp.json()
async def main():
urls = [f'https://api.example.com/items/{i}' for i in range(100)]
async with aiohttp.ClientSession() as session:
results = await gather_limited(
[fetch(session, url) for url in urls],
limit=5,
)
return results
Check if a file or directory exists
The modern, cross-platform way to check for a file's existence using pathlib instead of the older os.path functions.
How it works: pathlib.Path wraps filesystem paths in an object with .is_file()/.is_dir() methods, which is more readable and less error-prone than string-based os.path.isfile()/os.path.isdir() calls.
from pathlib import Path
def file_exists(path):
return Path(path).is_file()
def dir_exists(path):
return Path(path).is_dir()
# Example usage
if file_exists('config.json'):
print('Config found')
Flatten a nested list
Recursively flatten an arbitrarily nested list into a single flat list.
How it works: for each item, if it's itself a list the function recurses into it and extends the result; otherwise the item is appended as-is.
def flatten(items):
result = []
for item in items:
if isinstance(item, list):
result.extend(flatten(item))
else:
result.append(item)
return result
# Example usage
flatten([1, [2, 3, [4, 5]], 6]) # -> [1, 2, 3, 4, 5, 6]
Read a CSV file into a list of dictionaries
The standard way to parse a CSV file's header row into keys, using Python's built-in csv module — no pandas needed for simple cases.
How it works: csv.DictReader uses the first row as field names automatically, so each row comes back as a dict you can access by column name instead of a fragile numeric index.
import csv
def read_csv(path):
with open(path, newline='', encoding='utf-8') as f:
return list(csv.DictReader(f))
# Example usage
rows = read_csv('users.csv')
for row in rows:
print(row['name'], row['email'])
Retry Decorator with Backoff (Python)
A decorator for retrying unstable functions with exponential backoff and jitter. Configurable exceptions, attempt count, and delay.
How it works: @wraps(func) preserves the original function's metadata. On each failure the delay doubles (base_delay * 2 ** attempt), capped at max_delay. A 0–20% jitter spreads retries from many clients across time.
import time
import random
from functools import wraps
def retry(max_attempts=3, base_delay=0.2, max_delay=2.0, exceptions=(Exception,)):
"""Retry a call with exponential backoff and jitter."""
def decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
for attempt in range(1, max_attempts + 1):
try:
return func(*args, **kwargs)
except exceptions as exc:
if attempt >= max_attempts:
raise
delay = min(max_delay, base_delay * 2 ** (attempt - 1))
jitter = random.uniform(0, delay * 0.2)
time.sleep(delay + jitter)
return wrapper
return decorator
# Example usage
import requests
@retry(max_attempts=4, base_delay=0.5, exceptions=(requests.RequestException,))
def fetch_data(url: str) -> dict:
response = requests.get(url, timeout=5)
response.raise_for_status()
return response.json()
data = fetch_data('https://api.example.com/items')
Chunk Array into Groups
Splits an array into groups of a given size — useful for batch processing, in-memory pagination, or sending data in parts.
How it works: iterates over the array in steps of size, slicing a chunk at each step. The last chunk may be smaller than size if there aren't enough elements.
function chunk<T>(array: T[], size: number): T[][] {
if (size <= 0) throw new RangeError('chunk size must be > 0')
const chunks: T[][] = []
for (let i = 0; i < array.length; i += size) {
chunks.push(array.slice(i, i + size))
}
return chunks
}
// Example usage
chunk([1, 2, 3, 4, 5], 2) // [[1,2],[3,4],[5]]
chunk(['a','b','c','d'], 3) // [['a','b','c'],['d']]
// Batch Telegram sends (30 msg/sec limit)
const userBatches = chunk(userIds, 25)
for (const batch of userBatches) {
await Promise.all(batch.map((id) => sendMessage(id, text)))
await sleep(1000)
}
Group Array by Key
A lodash-free groupBy — groups an array of objects by a field value into a Record. TypeScript-typed, with a note on the native Object.groupBy alternative.
How it works: uses reduce to accumulate items into an object where keys are the grouping field values and values are arrays of matching elements. In modern environments (Node 21+) you can use the native Object.groupBy instead.
function groupBy<T>(array: T[], key: keyof T): Record<string, T[]> {
return array.reduce((groups, item) => {
const groupKey = String(item[key])
groups[groupKey] = [...(groups[groupKey] ?? []), item]
return groups
}, {} as Record<string, T[]>)
}
// Native alternative (Node 21+ / Chrome 117+)
// const grouped = Object.groupBy(array, (item) => item.status)
// Example usage
const orders = [
{ id: 1, status: 'paid' },
{ id: 2, status: 'pending' },
{ id: 3, status: 'paid' },
]
groupBy(orders, 'status')
// { paid: [{id:1,...},{id:3,...}], pending: [{id:2,...}] }
Pagination Utilities (Offset / Limit)
Typed helpers for server-side pagination — compute offset and limit from a page number, return metadata (totalPages, hasNext, hasPrev).
How it works: paginate converts page/perPage into offset/limit for a SQL query. paginationMeta takes the total count and returns all metadata for a consistent API response format.
interface PaginationInput {
page: number
perPage?: number
}
interface PaginationMeta {
total: number
page: number
perPage: number
totalPages: number
hasNext: boolean
hasPrev: boolean
}
function paginate({ page, perPage = 20 }: PaginationInput) {
const safePage = Math.max(1, Math.floor(page))
const limit = Math.max(1, Math.min(perPage, 100))
const offset = (safePage - 1) * limit
return { limit, offset, page: safePage }
}
function paginationMeta(total: number, input: PaginationInput): PaginationMeta {
const { page } = paginate(input)
const perPage = input.perPage ?? 20
const totalPages = Math.ceil(total / perPage) || 1
return { total, page, perPage, totalPages, hasNext: page < totalPages, hasPrev: page > 1 }
}
// Example usage — SQL query
const { limit, offset } = paginate({ page: 3, perPage: 10 })
// SELECT * FROM orders LIMIT 10 OFFSET 20
const meta = paginationMeta(135, { page: 3, perPage: 10 })
// { total: 135, page: 3, perPage: 10, totalPages: 14, hasNext: true, hasPrev: true }
useFetch with AbortController — TypeScript
A Vue composable for fetch requests that automatically cancels the previous request when a new one is made. Reactive data, loading, and error states included.
How it works: every new execute call first cancels the previous request via AbortController.abort(). This prevents race conditions — a stale response won't overwrite a fresh one. AbortError is ignored; all other errors are written to error.
import { ref } from 'vue'
export function useFetch<T>() {
const data = ref<T | null>(null)
const loading = ref(false)
const error = ref<Error | null>(null)
let controller: AbortController | null = null
async function execute(url: string, options: RequestInit = {}) {
// Cancel the previous request
controller?.abort()
controller = new AbortController()
loading.value = true
error.value = null
try {
const res = await fetch(url, { ...options, signal: controller.signal })
if (!res.ok) throw new Error(`HTTP ${res.status}: ${res.statusText}`)
data.value = await res.json()
} catch (e) {
if ((e as Error).name !== 'AbortError') {
error.value = e as Error
}
} finally {
loading.value = false
}
}
function abort() {
controller?.abort()
}
return { data, loading, error, execute, abort }
}
// Example usage
const { data: users, loading, error, execute } = useFetch<User[]>()
// Switching pages — the old request is cancelled automatically
watch(currentPage, (page) => {
execute(`/api/users?page=${page}`)
})
Zod API validation schemas
Define type-safe request validation with Zod — parse, transform, and validate API request bodies, query strings, and URL params.
How it works: Zod schemas define the expected shape and constraints of incoming data. parse throws on invalid data; safeParse returns a result object so you can handle errors gracefully without try/catch.
import { z } from 'zod'
const CreateUserSchema = z.object({
email: z.string().email(),
name: z.string().min(2).max(100),
age: z.number().int().positive().optional(),
role: z.enum(['user', 'admin']).default('user'),
})
// Use with safeParse for readable errors
const result = CreateUserSchema.safeParse(body)
if (!result.success) {
const messages = result.error.issues.map(
issue => `${issue.path.join('.')}: ${issue.message}`
)
return { ok: false, errors: messages }
}
// result.data is fully typed
createUser(result.data)
// Reusable fragments
const PaginationSchema = z.object({
page: z.coerce.number().int().positive().default(1),
limit: z.coerce.number().int().min(1).max(100).default(20),
})
const UuidParam = z.object({
id: z.string().uuid(),
})
// Express handler example
app.post('/api/users', (req, res) => {
const parsed = CreateUserSchema.safeParse(req.body)
if (!parsed.success) return res.status(400).json(parsed.error.flatten())
res.json(createUser(parsed.data))
})
More snippet collections
Contact Information
Submit a request or write to me directly on Telegram. I'll reply within 24 hours.