Skip to content
Back to blog

LLM Developer Workbench: RAG Chunking, API Cost and PII Scanning

3 min read
LLM Developer Workbench: RAG Chunking, API Cost and PII Scanning

Shipping an LLM feature comes with three problems that never show up in the demo. Your documents don't fit in a context window, so they need chunking before they reach a vector database. The API bill looks fine in testing and turns unpredictable at scale, because nobody priced the hosting it runs on top of. And a prompt pasted from a bug report or a support ticket has a way of carrying a credential nobody meant to send anywhere.

Chunking a document for retrieval

A vector database indexes chunks of text, not whole documents, so the first real decision in a RAG pipeline is how to cut a document into pieces without cutting the meaning along with it.

The RAG Text Chunker splits on a separator you choose — a blank line, a single line break, or a sentence boundary — and only falls back to a raw character cut when nothing safer is available. A unit that still has to be split at your size limit is broken at the nearest sentence boundary first, then the nearest whitespace, then a Unicode-aware word boundary for scripts that don't use spaces at all. That ordering matters: a chunk boundary that lands inside a word or mid-sentence degrades exactly the kind of semantic search a RAG pipeline exists to do well.

Overlap carries context across that cut. Instead of starting the next chunk from nothing, the tool prepends the tail of the previous chunk — cut, again, at a sentence or word boundary rather than an arbitrary character count — so a fact split across two chunks still has a chance of being retrieved by whichever one a query happens to match.

The token count shown next to each chunk is deliberately labeled estimated: it's a chars ÷ 4 heuristic, not a real tokenizer for any specific model. That's enough to judge whether a chunk fits a rough budget; it isn't enough to bill against, which is what the next tab is for.

Pricing an API call honestly

The Token & Cost Estimator does one job and says so plainly: it multiplies token counts you provide by a model's published rate. It doesn't tokenize your text and it doesn't call OpenAI, Anthropic or Google — the "Live price registry" badge tells you whether the current prices came from this site's own JSON feed or from a small bundled fallback, and every row links back to the provider's own pricing page with a reminder to check it, because published rates change without notice and a stale number in a spreadsheet is how a budget goes wrong quietly.

That estimator feeds a second, easy-to-forget number: hosting. A per-call LLM cost multiplied by monthly active users is the number most cost projections stop at, but a real deployment also pays for object storage, egress, and whatever serves the request — so the Monthly Cost Projection section adds AWS S3, Cloudflare R2, Vercel and Cloudflare Workers pricing, each billed the way that provider actually bills it (S3's separate storage/PUT/GET/egress-over-free-tier, Vercel's included-bandwidth-then-overage, and so on), driven by the same MAU number.

Catching a secret before it leaves the tab

The PII & Secrets Sanitizer scans a pasted prompt for the shapes leaked credentials actually take: sk-proj-… and other OpenAI key prefixes, AWS access keys, GitHub tokens, Slack bot tokens, Stripe secret keys, plus email addresses, phone numbers and SSN-shaped digit groups. Anything matched is redacted in the output and counted, so a prompt that looked clean at a glance shows exactly what it was actually carrying.

This is a heuristic pattern scan, not a security audit, and the page says so before you paste anything in. A credential that shows up here should be rotated regardless of what happens next — the tool finding it doesn't make it safe, it makes it visible.

Nothing leaves the browser

Chunking, cost math and pattern matching all run in page memory. The one network request the page makes on its own is fetching the public price registry JSON — no document text, token count or prompt content is ever part of that request.

Frequently asked questions

Does the RAG Text Chunker use a real tokenizer?

No — the token count it shows is an estimate using a chars ÷ 4 heuristic, not an exact tokenizer for any specific model. That's enough to size a chunk against a rough context budget; for a real per-call cost, use your provider's own tokenizer or the token count a real API response reports.

How does the chunker decide where to cut?

It tries hard not to cut mid-word or mid-sentence. Splitting first respects your chosen separator — paragraph, line or sentence breaks — and any unit still over the size limit is cut at the nearest sentence boundary, then the nearest whitespace, then a Unicode-aware word boundary for scripts without spaces. A raw character cut only happens as the last resort.

Where do the model and hosting prices come from?

A versioned JSON registry served from this site, with a small bundled fallback baked into the page in case the live fetch fails. Every entry carries the date it was last checked and a source_url to the provider's own pricing page — the tool shows that date and tells you to verify before budgeting, because published rates change without notice.

Does the cost estimator actually call OpenAI, Anthropic or Google?

No. You supply the input and output token counts — from the RAG Chunker's estimate, your own tokenizer, or a real API response — and it multiplies them by the selected model's published rate. It's a calculator against list prices, not a live API client.

What does the PII & Secrets Sanitizer actually catch?

Pattern-matched credentials with a recognizable shape: OpenAI keys, AWS access keys, GitHub tokens, Slack bot tokens, Stripe secret keys, email addresses, phone numbers and SSN-shaped digit groups. It's a heuristic scan, not a security audit — treat every credential it finds as compromised and rotate it regardless of what the tool reports.

Is anything uploaded to a server?

No document text, token counts or prompt content ever leave the browser. The one network request the page makes on its own is fetching the public price registry JSON, which carries no user data.

Author

Fullstack Engineer

In commercial development since 2011 — over 15 years. Author of two SaaS products, TeleGo.io and LingvoHabit.com. The articles here come out of real client and product work.