Build a BPE Tokenizer from Scratch in Python, Step-by-Step Guide
Build a working BPE tokenizer in Python step by step. Learn how LLMs split text into tokens, implement byte pair encoding, and count tokens with...
NLP
NLP tutorials from Machine Learning Plus: hands-on guides with worked examples and code you can read, copy and run. Start with the fundamentals and work through to the applied posts in the Generative AI and NLP track.
2026 2
How to Evaluate an LLM: Benchmarks, Metrics, and Practical Workflows
Evaluate your LLM using MMLU, MT-Bench, LLM-as-judge, and ROUGE. Covers lm-evaluation-harness, fine-tuned model comparison, and evaluation pitfalls...
2021 1
What is Tokenization in Natural Language Processing (NLP)?
Learn what tokenization is in NLP and why it matters. Covers word, subword, and character tokenization methods with practical Python examples.
2020 7
Text Summarization Approaches for NLP – Practical Guide with Generative Examples
Text Summarization is summarizing the information in large texts for quicker consumption. In this article, I will walk you through the extractive as...
Complete Guide to Natural Language Processing (NLP) – with Practical Examples
Natural language processing (NLP) is the technique by which computers understand the human language. NLP allows you to do text classification...
Building chatbot with Rasa and spaCy
This article guides you step-by-step on how to create a chatbot using Rasa open-source. Usage of Rasa NLU and Rasa Core is also explained.
SpaCy Text Classification – How to Train Text Classification Model in spaCy (Solved Example)?
Text Classification is the process of categorizing texts. SpaCy makes text classification convenient using the `textcat` component
spaCy Tutorial – Complete Writeup
spaCy is an advanced modern library for Natural Language Processing developed by Matthew Honnibal and Ines Montani. This tutorial is a complete guide...
101 NLP Exercises (using modern libraries)
This post provides solutions to all major NLP problems from basic use of packages to advanced text calssification, summarization ,translation , etc.
Training Custom NER models in SpaCy to auto-detect named entities [Complete Guide]
Named-entity recognition (NER) identifies the entities in a text and classifies them into categories such as person, organization and location. Learn...
2018 6
Topic modeling visualization – How to present the results of LDA models?
In this post, we follow a structured approach to build gensim's topic model and explore multiple strategies to visualize results using matplotlib...
Cosine Similarity – Understanding the math and how it works (with python codes)
Cosine similarity is a metric used to measure how similar the documents are irrespective of their size. It is the cosine of the angle between two...
Gensim Tutorial – A Complete Beginners Guide
Gensim is billed as a Natural Language Processing package that does 'Topic Modeling for Humans'. But it is practically much more than that. It is a...
Lemmatization Approaches with Examples in Python
Lemmatization is the process of converting a word to its base form. Python has nice implementations through the NLTK, TextBlob, Pattern, spaCy and...
LDA in Python – How to grid search best topic models?
Python's Scikit Learn provides a convenient interface for topic modeling using algorithms like Latent Dirichlet allocation(LDA), LSI and Non-Negative...
Topic Modeling with Gensim (Python)
Topic Modeling is a technique to understand and extract the hidden topics from large volumes of text. Latent Dirichlet Allocation(LDA) is an...