qkarakter

2. Convert to Lower Case by Considering Punctuation Marks:

Example:

POLIOMYELITIS (POLIO) IS A HIGHLY INFECTIOUS VIRAL DISEASE, WHICH MAINLY AFFECTS YOUNG CHILDREN. THE VIRUS IS TRANSMITTED THROUGH CONTAMINATED FOOD AND WATER, AND MULTIPLIES IN THE INTESTINE, FROM WHERE IT CAN INVADE THE NERVOUS SYSTEM. MANY INFECTED PEOPLE HAVE NO SYMPTOMS, BUT DO EXCRETE THE VIRUS IN THEIR FAECES, HENCE TRANSMITTING INFECTION TO OTHERS.

Poliomyelitis (polio) is a highly infectious viral disease, which mainly affects young children. The virus is transmitted through contaminated food and water, and multiplies in the intestine, from where it can invade the nervous system. Many infected people have no symptoms, but do excrete the virus in their faeces, hence transmitting infection to others.
3. Remove to Unnecessary Space and Tabs

Example:

Before:

It is now generally acknowledged that the global climate is changing, as the earth becomes warmer.

This     change has the potential to   affect human health in a number of ways,     for instance by altering the geographic range and seasonality of certain infectious diseases,         disturbing food-producing   ecosystems,    and increasing the frequency of extreme weather events,    such as hurricanes.

After:

It is now generally acknowledged that the global climate is changing, as the earth becomes warmer.
This change has the potential to affect human health in a number of ways, for instance by altering the geographic range and seasonality of certain infectious diseases, disturbing food-producing ecosystems, and increasing the frequency of extreme weather events, such as hurricanes.
4. ISO – UTF Problems (Espectially Turkish Charcode mainly problems of forwarded emails and pdf to word converted docs…)

Example:

Before:

Pardus Nedir?
Pardus, T�BİTAK-UEKAE (Ulusal Elektronik ve Kriptoloji Ara�tırma Enstitüsü) bünyesinde yürütülen bir i�letim sistemi geli�tirme projesidir. Pardus açık kaynak kodlu ve GPL (GNU Genel Kamu Lisansı) ile da�ıtılan bir özgür yazılımdır.

After:

Pardus Nedir?
Pardus, TüBİTAK-UEKAE (Ulusal Elektronik ve Kriptoloji Araştırma Enstitüsü) bünyesinde yürütülen bir işletim sistemi geliştirme projesidir. Pardus açık kaynak kodlu ve GPL (GNU Genel Kamu Lisansı) ile dağıtılan bir özgür yazılımdır.
5. Remove Forward (>>) Punctuation Mark

6. Convert to English Alphabet:

Convert to Turkish, French, German specific characters to similar english characters.

Example:

Ş ç Ü ü À Â Ç Ñ -> S c U u A A C N
7. Format:

Convert to Lower Case by considering Punctuation Marks
Remove to unecessary spaces and tabs.
Re-format to spaces by considering after and before punctuation marks

Before:

this change          has the potential to affect Human Health in a number of ways   , for instance by altering the geographic range and seasonality of certain infectious Diseases ,    disturbing food-producing ecosystems ,and increasing the frequency of extreme weather events ,such as hurricanes .

After:

This change has the potential to affect human health in a number of ways, for instance by altering the geographic range and seasonality of certain infectious diseases, disturbing food-producing ecosystems, and increasing the frequency of extreme weather events, such as hurricanes.
8. Replace/Change words