NLTK Essentials [BOOK, 2015]

Posted by Franz

NLTK Essentials
July 27, 2015
by Nitin Hardeniya
https://www.amazon.com/NLTK-Essentials-Nitin-Hardeniya/dp/1784396907

Mastering Natural Language Processing with Python
Deepti Chopra, Nisheeth Joshi, Iti Mathur
June 2016
https://www.packtpub.com/big-data-and-business-intelligence/mastering-natural-language-processing-python
https://www.amazon.com/Mastering-Natural-Language-Processing-Python/dp/1783989041

https://www.packtpub.com/big-data-and-business-intelligence/natural-language-processing-python-and-nltk

Text Analytics with Python
A Practical Real-World Approach to Gaining Actionable Insights from your Data
Authors: Sarkar, Dipanjan
http://www.apress.com/us/book/9781484223871

http://pyparsing.wikispaces.com
http://infohost.nmt.edu/tcc/help/pubs/pyparsing/web/index.html
2013

https://www.amazon.com/Web-Scraping-Python-Collecting-Modern/dp/1491910291

Python Projects for Kids
2016
https://www.amazon.com/dp/1782175067

Java Programming for Kids
2014
https://www.amazon.com/dp/B00O9GAGYQ

NLTK word_tokenize()

Posted by Franz

cell_structure NLTK word_tokenize()
http://www.nltk.org/book/ch03.html#tokenization_index_term

>>>from nltk import word_tokenize

>>>word_tokenize(‘cells’ structure’)
[‘cells’‘, ‘structure’]

>>with open(‘h.txt’) as f:
____word_tokenize(f.read())

Source code for nltk.collocations()
http://www.nltk.org/api/nltk.html
http://www.nltk.org/_modules/nltk/collocations.html

>>> dir(nltk.collocations)

>>> print(nltk.collocations.__doc__)

>>> with open(‘Histology14_Ch01_ALL.txt’) as f:
____nltk.Text(word_tokenize(f.read())).collocations()

cell nuclei;
electron microscopy;
fluorescent compounds;
glass slides;
gold particles;
labeled secondary; <=======
light microscope;
light microscopy;
MEDICAL APPLICATION;
nucleic acids;
objective lens;
organic solvents;
primary antibody;
resolving power;
secondary antibody;
secretory granules;
situ hybridization; <=======
tissue components;
tissue section;
tissue sections

>> with open(‘Histology14_Ch01_i.txt‘) as f:
nltk.Text(word_tokenize(f.read())).collocations()

matrix components;
tissue biology

Naturally, the quality of the collocations is also higher than computer-generated lists – as we would expect from a manually produced compilation.
p. 174
http://nlp.stanford.edu/fsnlp/promo/colloc.pdf
phrasal verbs: good example of a collocation with often non-adjacent words

related:
Collocations dictionary
https://franzcalvo.wordpress.com/2015/09/07/collocations

Intro to NLP (UM 2015)

Posted by Franz

Introduction to Natural Language Processing
University of Michigan
Coursera, October 5 – December 27, 2015
https://www.coursera.org/course/nlpintro

Greedy transition-based parsing
https://class.coursera.org/nlp/lecture/177

NLTK dependency parsing
https://github.com/nltk/nltk/wiki/Dependency-Parsing
https://github.com/nltk/nltk/blob/develop/nltk/parse/transitionparser.py

NLTK BOOK
6. Learning to Classify Text
http://www.nltk.org/book/ch06.html

Assignment 1
https://class.coursera.org/nlpintro-001/forum/thread?thread_id=3 Feature extracting question
https://class.coursera.org/nlpintro-001/forum/thread?thread_id=143
Ms. Haag plays Elianti

Assignment on word similarity
https://class.coursera.org/nlpintro-001/forum/thread?thread_id=59 https://class.coursera.org/nlpintro-001/forum/thread?thread_id=59#post-244

Assignment 2, part 3A
https://class.coursera.org/nlpintro-001/forum/thread?thread_id=163#post-632

the assignments for this class have been developed and tested on Python 2.7 and NLTK 2.

DLNLP
http://web.eecs.umich.edu/~radev/dlnlp/list.txt
—————————————
CS224d: Deep Learning for Natural Language Processing
March-June 2015
http://cs224d.stanford.edu
—————————————

International Workshop on Semantic Evaluation 2015
http://alt.qcri.org/semeval2015

SDP 2015: Broad-Coverage Semantic Dependency Parsing
http://alt.qcri.org/semeval2015/task18

some good papers in NLP
https://class.coursera.org/nlpintro-001/forum/thread?thread_id=48#post-211
NLP libraries in Java
https://class.coursera.org/nlpintro-001/forum/thread?thread_id=33#post-128

this course is more introductory than …
https://class.coursera.org/nlpintro-001/forum/thread?thread_id=46#post-197

volunteers: covering installation of Python and NLTK on different platforms

http://www.ioling.org

https://www.linkedin.com/in/sabinebuchholz

nltk.compat

Posted by Franz

NLTK_ImportError

Miniconda_3.9.1_installed

NLTK.compat_is_installed

scikit-learn 0.15.2

Posted by Franz

scikit-learn
http://scikit-learn.org

0.15.2
>>> sklearn.__version__
https://pypi.python.org/pypi/scikit-learn/0.15.2

dependencies
scikit-learn is tested to work under Python 2.6, Python 2.7, and Python 3.4.
The required dependencies to build the software are

NumPy >= 1.6.2
>>> numpy.__version__
https://pypi.python.org/pypi/numpy/1.10.1 (type: Python Wheel)http://www.numpy.org (Installing the SciPy Stack …)
SciPy >= 0.9
>>> scipy.__version__
http://www.scipy.org
ScyPy stack http://www.scipy.org/install.html
You can also build any of the SciPy packages from source, for instance if you want to get involved with development. This is easy for packages written entirely in Python, while others like NumPy require compiling C code.
- Anaconda 392 Mb
  Miniconda (4 Mb)
  http://repo.continuum.io/miniconda/index.html
- Enthought Canopy 300 Mb
- Python(x,y) (It is recommended to uninstall any other Python distribution before installing Python(x,y))
- WinPython 267 Mb

a working C/C++ compiler.

~~Create a separate environment~~
~~http://conda.pydata.org/docs/using/envs.html~~

—————————-

NumPy (4.6 MB) download
http://sourceforge.net/projects/numpy/files
the below notes are about building Numpy, which for most users is *not* the recommended way to install Numpy. Instead, use either a complete scientific Python distribution or a binary installer

——————————–

Dragomir Radev, September 2015
Here is how to install a specific older version of a Python library:
pip uninstall scikit-learn
or
pip uninstall sklearn
then
pip install scikit-learn==0.15.2

Hint: the following packages conflict with each other:
– scikit-learn ==0.15.2
– python 3.5*

conda

Conda
http://conda.pydata.org
© Continuum Analytics

.tar.gz files
http://www.gzip.org
http://www.7-zip.org

Assignment on word similarity
https://class.coursera.org/nlpintro-001/forum/thread?thread_id=59 https://class.coursera.org/nlpintro-001/forum/thread?thread_id=59#post-244

International Workshop on Semantic Evaluation 2015
http://alt.qcri.org/semeval2015

SDP 2015: Broad-Coverage Semantic Dependency Parsing
http://alt.qcri.org/semeval2015/task18

Assignment 2, part 3A
https://class.coursera.org/nlpintro-001/forum/thread?thread_id=163#post-632

this course is more introductory than …
https://class.coursera.org/nlpintro-001/forum/thread?thread_id=46#post-197

the assignments for this class have been developed and tested on Python 2.7 and NLTK 2.

volunteers: covering installation of Python and NLTK on different platforms

http://www.ioling.org

Stemming

Posted by Franz

Morphological similarity: Stemming

Stemming and Lemmatization with Python NLTK
http://text-processing.com/demo/stem

Martin Porter’s official site:
http://tartarus.org/martin/PorterStemmer

WordNet::Similarity

Posted by Franz

WordNet::Similarity
http://wn-similarity.sourceforge.net
a Perl module that implements a variety of semantic similarity and relatedness measures based on information found in the lexical database WordNet.

WordNet
A lexical database for English
https://wordnet.princeton.edu

http://marimba.d.umn.edu/cgi-bin/similarity/similarity.cgi

UMLS::Similarity v1.41 released! (July 17, 2014)
http://www.d.umn.edu/~tpederse

North American Chapter of the Association for Computational Linguistics
http://naacl.org

NLTK_similarity NLTK (Python)

challenge:
How Strong Is Your Vocabulary?
http://www.merriam-webster.com/quiz/index.htm

from:
Introduction to Natural Language Processing
University of Michigan
Coursera, October 5 – December 27, 2015
https://www.coursera.org/course/nlpintro

Python Regular Expressions

Posted by Franz

The Python Standard Library > 6. Text Processing Services > re
https://docs.python.org/3.4/library/re.html

Regular expressions HOWTO:
https://docs.python.org/3.4/howto/regex.html
For … science underlying regular expressions (deterministic and non-deterministic finite automata), you can refer to almost any textbook on writing compilers.

Metacharacters are not active inside classes: … '$' is usually a metacharacter, but inside a character class it’s stripped of its special nature.

Perhaps the most important metacharacter is the backslash, \.

Some of the special sequences beginning with '\' represent predefined sets of characters

* doesn’t match the literal character *; instead, it specifies that the previous character can be matched zero or more times, instead of exactly once.

Pay careful attention to the difference between * and +;

match() versus search()

========================================

Regular expressions
http://www.pythonlearn.com/html-008/cfbook012.html

Python Regular Expressions
2012
https://developers.google.com/edu/python/regular-expressions

Christopher Potts emoticons
https://class.coursera.org/nlp/lecture/145

Python: Regular expressions
University of Cambridge
http://www.ucs.cam.ac.uk/docs/course-notes/unix-courses/archived/archived-python-courses/PythonRE/files/PythonRE.pdf

regular123

Python 3 Text Processing with NLTK 3 Cookbook

Posted by Franz

Python 3 Text Processing with NLTK 3 Cookbook
August 26, 2014
by Jacob Perkins
http://www.amazon.com/Python-Text-Processing-NLTK-Cookbook/dp/1782167854

Python Text Processing with NLTK 2.0 Cookbook
November 11, 2010
by Jacob Perkins
http://www.amazon.com/Python-Text-Processing-NLTK-Cookbook/dp/1849513600

NLTK

Posted by Franz

NLP with Python [BOOK]
http://www.nltk.org/book
By Steven Bird, Ewan Klein, Edward Loper
O’Reilly Media
June 2009

http://www.nltk.org
The Natural Language Toolkit (NLTK) is a Python package for
natural language processing. NLTK requires Python 2.6, 2.7, or 3.2+.
Author: Steven Bird

CHILDES Corpus Readers
http://www.nltk.org/howto/childes.html
Count the number of words and sentences of each file.

Chat-80
http://www.nltk.org/howto/chat80.html
The (deliberately naive) grammar sql.fcfg translates from English to SQL:
What cities are in China?