Natural Language Processing:A Brief Agenda to Get You Started

Posted by

Recently I start to tutor one paper (COMP700 Text and Vision Intelligence) for two AUT students.

This paper mostly involves in Computer Vision and Text Mining (Natural Language Processing).

The two aspects are common applications about Aritificial Intelligence.

So, in this blog, I will provide a brief agenda about Natural Language Processing for people who want to get kick-start in Artificial Intelligence area.

General Intro NLP
~General overview
~Tasks to describe languages

Language model:
~Statistical model
~Neural network model

NLP with deep learning:
~Recurrent Neural Network
~Convolutional Neural Network

NLP at different levels:
~Phonetic/Phonological Analysis
~OCR
~Morphological analysis
~Syntactic analysis
~Semantic interpretation
~Discourse processing

NLP in industry:
~Search
~Automated/assisted translation
~Speech recognition
~Sentiment analysis
~Chatbot

What need to be done-representing:
~Morphology
~Sentence structure
~Senmantic
~Vectors

Word vector:
~Wordnet: a resource containing lists of synonym sets and hypernyms
~One-hot: as discrete symbols
~TF-IDF
~Word embedding: A word’s meaning is given by the words that frequently appear close-by

Knowledge graph-semantic drive way:
~Ontologies&Description logic
~OWL&RDF
~Semantic web
~Dgraph, Neo4j

Describing Language tasks:
~Segment(for Chinese)
~POS: Part Of Speech
~NER: Named Entity Recognition
~Algorithml:Rule-based taggers, Probabilistic tagger: HMM and Veterbi, Perceptron, Conditional model: CRF

Statistical Language Model:
~NNLM 2003
~RNNLM 2010
~CBoW: Continuous Bag-of-Words Model
~Skip-gram
~Word2vec

Neural Network:
~Neuron
~Activation function
~Back propagation: What’s cost function, How cost function used updated parameters

Activation function:
~Sigmoid
~Tanh
~ReLu (rectified lieaner)
~Softmax
~…

Deep Learning:
~Learn (multiple levels of) representation and an output from ‘raw’ inputs x
~Universal, learnable framework for representing world, visual and linguistic information
~Can learn unsupervised (from raw text) and supervised
~Why now popular:A large dataset, Faster machines and multicore CPU/GPU
~Why it works

Deep Learning Models:
~Feed-Forward Networks
~Recurrent Neural Networks: simple RNN, LSTM, GRU
~Generative Neural Networks

If you are interested in some high income skills or have any problems with Natural Language Processing, feel free to contact me.

Or you can connect with me through my LinkedIn.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s