• Happiness is not achieved by the conscious pursuit of happiness; it is generally the by-product of other activities
• “Laughter and tears are both responses to frustration and exhaustion. I myself prefer to laugh, since there is less cleaning up to do afterward.”
• The road to wisdom?
  Well its plain and simple to express:
  Err and err and err again,
  but less and less and less. — Piet Hein

• Knowledge shared ,..sprouts..springs….blossoms and spreads fragrance..
  Knowledge withheld..Stagnates and stinks !!  - kris chaitu

  Slap or a Praise….
  (I give and take … )
  Straight on the face… – kris chaitu

picked up these two from a photonics researcher’s orkut profile.

So Long farewell .. Auf Weidersehen .. Adieu .. Adieu . To you !!
The time has come and the world is awaiting you !!
So bid you farewell .. au revoir !!
Wishing you success .. happiness .. health .. wealth .. accolades .. aplenty .. in abundance ..
Take very good care of yourself !!
Was lovely having you around and will miss you..

Id like to thank you guys for this

HELMET So, Lone Starr, Yogurt has taught you well. If there is one thing I despise, it is a fair fight. But if I must than I must. May the best man win. Put ‘er there. (offers to shake his hand)

LONE STARR goes to shake his hand. HELMET takes the ring off LONE STARR’S hand.

HELMET The ring. I can’t believe you fell for the oldest trick in the book. What a goof. What’s with you man? Come on. You know what? No, here let me give it back to you. (offers the ring back)

LONE STARR goes up to get the ring back. HELMET throws it in a grate. The ring goes in the grate. LONE STARR tries to catch it and falls to the grate.

HELMET Oh, look. You fell for that, too. I can’t believe it man.

LONE STARR gets up and runs to a corner.

HELMET So, Lone Starr, now you see that evil will always triumph, because good is dumb.

lolz.. id like to be evil now..  :P

I hope this post will be helpful to anyone who wants to do the same, i.e, make a local copy of Wikipedia’s database.

Things required

• First you should be able to decide what data you want from wikipedia and download the corresponding tables accordingly. The mostly commonly preferred knowledge, however, is the page content. and this can be obtained by downloading ‘pages-latest-articles.xml.gz’ file . This dump populates three tables , i.e, page, revision, text . The latest Wikipedia dumps are available at Wiki Dumps . The other prominent tables include pagelinks and categorylinks. A complete description of the Wikipedia Schema is available at WikiSchema.
• There are two kinds of dumps given by Wikipedia , sql dumps and xml dumps. SQL dumps can directly be uploaded to the database using the following command (similar in windows and linux)

mysql -u username -ppassword database < filename
Note that there is no space between ‘-p’ and ‘password’. And make sure the comeplete path of the sql file is given.

• The xml files however can be converted to sql files using a tool mwdumper which is written in java. An example of using mwdumper is as follows.

java -jar mwdumper.jar enwiki-latest-pages-articles.xml -format=sql:1.5 > enwiki-latest-pages-articles-dump.sql

• Once we get the sql files we cannot directly dump them into the database because the dump file page-articles contains a set of insert statements only. So the database schema should be loaded prior to executing the dumping of page-articles . This schema file is available at WikiSchema . Download the tables.sql file and create the schema as follows.

mysql -u username -ppassword database < tables.sql

NOTE:  a good point while uploading pagelinks table.  
 

My preparation for GRE seemed worfully inadequate a few days ago, but now I think I struck a cord. Things began to make sense to me. My preparation got a rythm and now I feel I could make it possible, a score of atleast 1400. For TOEFL iBT I’m aiming for something above 100. Once I reach these goals, I know my self-confidence will be back.

The coming few weeks leading upto Oct 11 will be really tough for me. On one side, the mid semester examinations would eat-away nearly 1 full week. On the other hand, all my friends placed in “3rd round” have planned up a host of parties, which are pretty hard to ignore. Their celebrations really disturb me a lot more than any thing, but I have to learn to cope with it and not let  my concentration waver.

Ill be back soon after Toefl. I would like you to wish my mind all the self-confidence it can get.

A List of Available POS taggers on the Internet

Stanford Log-linear Part-Of-Speech Tagger
This POS Tagger is based on Maximum Entropy Markov models. It demonstrates the following ideas:
(i) Explicit use of both preceding and following tag contexts via a dependency network representation,
(ii) Broad use of lexical features, including jointly conditioning on multiple consecutive words,
(iii) Effective use of priors in conditional log-linear models
(iv) Fne-grained modeling of unknown word features.
Using these ideas together, the resulting tagger gave a 97.24% accuracy on the Penn Treebank WSJ, an error reduction of 4.4% on the best previous single automatically learned tagging result.
Link: Stanford Part-Of-Speech Tagger 
References:
[-] Kristina Toutanova and Christopher D. Manning. Enriching the Knowledge Sources Used in a Maximum Entropy Part-of-Speech Tagger. Proceedings of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora (EMNLP/VLC-2000), Hong Kong.
[-] Kristina Toutanova, Dan Klein, Christopher Manning, and Yoram Singer. 2003. Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network. In Proceedings of HLT-NAACL 2003 pages 252-259.

A part-of-speech tagger for English
This tagger uses a bidirectional inference algorithm for part-of-speech tagging. It is based on Maximum Entropy Markov Models (MEMM). The algorithm can enumerate all possible decomposition structures and find the highest probability sequence together with the corresponding decomposition structure in polynomial time. Experimental results of this part-of-speech tagger show that the proposed bidirectional inference methods consistently outperform unidirectional inference methods and bidirectional MEMMs give comparable performance to that achieved by state-of-the-art learning algorithms including kernel support vector machines.

Link: A part-of-speech tagger for English
References:
[-]Yoshimasa Tsuruoka and Jun’ichi Tsujii, Bidirectional Inference with the Easiest-First Strategy for Tagging Sequence Data, Proceedings of HLT/EMNLP 2005, pp. 467-474.

TnT — Statistical Part-of-Speech Tagging
Trigrams’n'Tags (TnT) is an efficient statistical part-of-speech tagger. This tagger is based on Hidden Markov models (HMM) and performs at least as well as other current approaches, including the Maximum Entropy framework. Along with Hidden Markov Models as a base system, some optimization techniques are used for smoothing and for handling unknown words.
Link:   TnT — Statistical Part-of-Speech Tagging  
References:
[-] Thorsten Brants, 2000. TnT – A Statistical Part-of-Speech Tagger. In Proceedings of the Sixth Applied Natural Language Processing Conference ANLP-2000, Seattle, WA.

ACOPOST – A Collection Of POS Taggers
ACOPOST is a set of freely available POS taggers that I modelled after well-known techniques. The programs are written in C and run under various UNIX flavors (and probably even under Windows). ACOPOST currently consists of four taggers which are based on different frameworks:
Maximum Entropy Tagger MET: This tagger uses an iterative procedure to successively improve parameters for a set of features that help to distinguish between relevant contexts. It’s based on a framework suggested by Ratnaparkhi [1997].

Trigram Tagger T3: This kind of tagger is based on Hidden Markov Models where the states are tag pairs that emit words, i. e., it’s based on transitional and lexical probabilities. The technique has been suggested by Rabiner [1990] and the implementation is influenced by Brants [2000].
Error-driven Transformation-based Tagger TBT: Transformation rules are learned from an annotated corpus which change the currently assigned tag depending on triggering context conditions. The general approach as well as the application to POS tagging has been proposed by Brill [1993].
Example-based tagger ET: Example-based models (also called memory-based, instance-based or distance-based) rest on the assumption that cognitive behavior can be achieved by looking at past experiences taht resemble the current problem rather than learning and applying acstract rules. They have been suggested for NLP by Daelemans et al. [1996].
Link:   ACOPOST  
References:
[-] Ingo Schröder. 2002. A Case Study in Part-of-Speech tagging Using the ICOPOST Toolkit. Technical report FBI-HH-M-314/02. Department of Computer Science, University of Hamburg. Available from http://nats-www.informatik.uni-hamburg.de/~ingo/papers/.
[-] Thosrten Brants. 2000. TnT – as statistical part-of-speech tagger. In Proceedings of the Sixth Applied Natural Language Processing Conference (ANLP-2000), Seattle, WA, USA.
[-] Eric Brill. 1993. Automatic grammar induction and parsing free text: A transformation-based appraoch. In Proceedings of the 31st Annual Meeting of the ACL.
[-] Walter Daelemans, Jakub Zavrel, Peter Berck & Steven Gillis. 1996. MBT: A memory-based part of speech tagger-generator. In Eva Ejerhed & Ido Dagan, ed., Proceedings of the Fourth Workshop on Very Large Corpora, pages 14-27.
[-] Adwait Ratnaparkhi. 1998. Maximum Entropy Models for Natural Language Ambiguity Resolution. Ph.D. thesis, University of Pennsylvania.

CLAWS part-of-speech tagger for English
One of the earliest probabilistic taggers was CLAWS (Constituent Likelihood Automatic Word-tagging System), developed by UCREL at the University of Lancaster. This POS tagging software for English text, CLAWS has been continuously developed since the early 1980s. The latest version of the tagger, CLAWS4, was used to POS tag c.100 million words of the British National Corpus (BNC). CLAWS4 could now be considered to be a hybrid tagger, involving both probabilistic and rule-based elements. It has been designed so that it can be easily adapted to different types of text in different input formats.
CLAWS has consistently achieved 96-97% accuracy (the precise degree of accuracy varying according to the type of text). Judged in terms of major categories, the system has an error-rate of only 1.5%, with c.3.3% ambiguities unresolved, within the BNC. More detailed analysis of the error rates for the C5 tagset in the BNC can be found within the BNC manual.
For more information on the CLAWS tagger, see Garside (1987), Leech, Garside and Bryant (1994), Garside (1996), and Garside and Smith (1997):
Link:  CLAWS
References:
[-] Garside, R. (1987). The CLAWS Word-tagging System. In: R. Garside, G. Leech and G. Sampson (eds), The Computational Analysis of English: A Corpus-based Approach. London: Longman.
[-] Leech, G., Garside, R., and Bryant, M. (1994). CLAWS4: The tagging of the British National Corpus. In Proceedings of the 15th International Conference on Computational Linguistics (COLING 94) Kyoto, Japan, pp622-628. (partial html version pdf version)
[-] Garside, R. (1996). The robust tagging of unrestricted text: the BNC experience. In J. Thomas and M. Short (eds) Using corpora for language research: Studies in the Honour of Geoffrey Leech Longman, London, pp 167-180.
[-] Garside, R., and Smith, N. (1997) A hybrid grammatical tagger: CLAWS4, in Garside, R., Leech, G., and McEnery, A. (eds.) Corpus Annotation: Linguistic Information from Computer Text Corpora. Longman, London, pp. 102-121.

Brill Tagger
In Brill(1992) a trainable rule-based tagger was described that obtained performance comparable to that of stochastic taggers, but captured relevant linguistic information in a small number of simple non-stochastic rules. In Brill-Tagger a number of extensions to this rule-based tagger are proposed. It describes a method for expressing lexical relations in tagging that stochastic taggers are currently unable to express. It implements a rule-based approach to tagging unknown words. It demonstrates how the tagger can be extended into a k-best tagger, where multiple tags can be assigned to words in some cases of uncertainty.
Link: Brill Tagger
References:
[-] Brill E. 1992, A simple rule-based part of speech tagger, In Proceedings of the Third Conference on Applied Natural Language Processing, ACL.
[-]Brill E. 1994, Some advances in rule-based part of speech tagging. In Pro-. ceedings of the Twelfth National Conference on Artificial Intelligence (AAAI-94), Seattle

Tree-Tagger – a language independent part-of-speech tagger
Tree-Tagger presents a new probabilistic tagging method which avoids problems that Markov Model based methods generally face, when they have to estimate transition probabilities from sparse data. Here transition probabilities are estimated using a decision tree. The decision tree automatically determines the appropriate size of the context which is used to estimate the transition probabilities.Tree-tagger is implemented based on this method and reported accuracies above 96% on Penn-Treebank WSJ corpus.
Link: Tree-Tagger 
References:
[-] H. Schmid. Probabilistic part-of-speech tagging using decision trees. In Proceedings of International Conference on New Methods in Language Processing, September 1994
[-] H. Schmid. Improvements in part-of-speech tagging with an application to German. In: Feldweg and Hinrichs, eds., Lexikon und Text, pp. 47-50