Sinhala Unicode Cleaner

Clean Unicode without converting it to a legacy font

This tool is for text that is already Sinhala Unicode. It applies Unicode normalization and removes selected invisible formatting problems. It does not turn Unicode into FM Abhaya, DL-Manel or another legacy encoding.

Normalization is useful after copying text between editors, websites and documents that may introduce different but canonically equivalent character sequences.

ZWJ and ZWNJ need careful handling

Zero-width joiner and zero-width non-joiner characters can affect how Indic-script sequences behave. Removing them blindly can change rendering or orthographic intent. That is why LankaTextTools keeps ZWJ/ZWNJ removal optional rather than treating every zero-width character as an error.

What the default cleanup changes

The default cleaner applies NFC normalization, standardizes non-breaking spaces and removes soft hyphen characters. It also trims repeated horizontal spaces while preserving line breaks. You can separately choose whether to remove ZWJ and ZWNJ characters.

Where hidden characters come from

Copying from web pages, PDFs, word processors and messaging apps can introduce non-breaking spaces, soft hyphens or zero-width characters. Some are intentional and some are artifacts of layout or line wrapping. Hidden characters can affect search, equality checks, database matching and character limits even when the text looks unchanged.

NFC normalization and Sinhala text

Unicode allows some text to have more than one canonically equivalent representation. NFC normalization aims to put equivalent sequences into a consistent composed form where the standard defines one. It does not translate words, fix spelling or convert legacy FM text.

Normalization is especially useful before comparing strings or storing text from several sources, because visually identical input is easier to handle when its Unicode representation is consistent.

When not to remove joiners

A zero-width joiner can influence how a consonant sequence is shaped. If you are cleaning text simply because a counter reports hidden code points, do not remove joiners automatically. Compare rendering before and after and understand what the destination system expects.

The optional switch exists for workflows that explicitly need joiner-free text, such as testing, debugging or a known downstream system limitation.

Use cleanup after conversion, not before every time

If a legacy converter relies on exact source codes, changing the legacy string before conversion can destroy information. Convert the legacy encoding first. Then use the Unicode cleaner on the Unicode result only when you have a reason to normalize spacing or joiner behavior.