Text processing options

Sun, 25 Apr 2010 00:00:00 +0000

The last two weeks, I have been researching options for processing free text. I think I have explored the entire spectrum of possibilities. Below are some notes that I can revisit in a few months and not spend the same effort again.

Background

I was looking at a way to process auto-generated tweets, like the ones on http://twitter.com/moneyvidya_com. Some sample tweets:

#moneyvidya arunthestocksguru (5 Star rated) says Buy Vijay Shanthi Builders - 6m (Monday 29 March 2010 @ 09:55 … http://bit.ly/bd5JgC

#moneyvidya arunthestocksguru (5 Star rated) says Buy Bhagwati Banquets & Hotels - 6m (Monday 29 March 2010 @ 09… http://bit.ly/9MzRDG

#moneyvidya NSV (5 Star rated) says Buy ACC - 6m (Wednesday 24 March 2010 @ 09:55 AM): http://bit.ly/b5xTrN

#moneyvidya justsurjit (5 Star rated) says Sell Sesa Goa - Intraday (Monday 22 March 2010 @ 10:31 AM): http://bit.ly/9lLo8U

As it is clear, the text follows a specific format, but has its own little variations. I intended to process the ‘insights’ and see each expert’s success rate. Although I never got around actually completing the task, I did learn a lot about text processing.

Lexical-Analysis on Dr. Hitesh Sarda

Text processing options

Background