Python Regular Expressions Tutorial and Examples: A Simplified Guide

Regular expressions (regex) let you search, extract, and manipulate text patterns in Python. They power text validation, NLP projects, and text mining workflows.

Regular Expressions in Python: A Simplified Tutorial. Photo by Sarah Crutchfield.Regular Expressions in Python: A Simplified Tutorial. Photo by Sarah Crutchfield.

This post has interactive code, click ‘Run’ or press Ctrl+Enter on any code block to execute it directly in your browser. The first run may take a few seconds to initialize.

1. Contents

  1. Introduction to regular expressions
  2. What is a regex pattern and how to compile one?
  3. How to split a string separated by a regex?
  4. Finding pattern matches using findall, search and match
  5. What does re.findall() do?
    5.1. re.search() vs re.match()
  6. How to substitute one text with another using regex?
  7. Regex groups
  8. What is greedy matching in regex?
  9. Most common regular expression syntax and patterns
  10. Regular Expressions Examples
    10.1. Any character except for a new line
    10.2. A period
    10.3. Any digit
    10.4. Anything but a digit
    10.5. Any character, including digits
    10.6. Anything but a character
    10.7. Collection of characters
    10.8. Match something upto ‘n’ times
    10.9. Match 1 or more occurrences
    10.10. Match any number of occurrences (0 or more times)
    10.11. Match exactly zero or one occurrence
    10.12. Match word boundaries
  11. Practice Exercises
  12. Conclusion

1. Introduction to regular expressions

Python implements regex through its built-in re module. You will find regex in NLP, web apps that validate input (like email addresses), and most text mining projects.

This tutorial has two parts. First, you will learn the five main features of the re module. Then you will see how to build common regex patterns in Python.

By the end, you will know how to construct almost any string pattern you need for text mining work.

2. What is a regex pattern and how to compile one?

A regex pattern is a special language that represents generic text, numbers, or symbols. You use it to extract text that fits the pattern.

Take the basic example '\s+'. Here '\s' matches any whitespace character. The '+' at the end makes it match one or more spaces.

This pattern also matches tab characters ('\t'). A full list of regex patterns appears at the end of this post. First, let’s see how to compile and use regular expressions.

PyInteractive Python
import re regex = re.compile('\s+') print("Compiled:", regex) print("Pattern:", regex.pattern)

  

The code above imports re and compiles a pattern that matches one or more whitespace characters.

3. How to split a string separated by a regex?

Consider this piece of text with three course items:

PyInteractive Python
import re text = """101 COM Computers 205 MAT Mathematics 189 ENG English""" print(text)

  

Each line follows the format: “[Course Number] [Course Code] [Course Name]”. The spacing between words is uneven. You want to split these into individual words.

You can split in two ways:

1. Use the re.split method.
2. Call the split method on a compiled regex object.

PyInteractive Python
import re text = """101 COM Computers 205 MAT Mathematics 189 ENG English""" regex = re.compile('\s+') # Both methods produce the same result print(re.split('\s+', text)) print(regex.split(text))

  

Both methods work. Which should you use? If you plan to reuse a pattern, compile it first. That avoids recompiling the same pattern over and over.

4. Finding pattern matches using findall, search and match

Suppose you want to extract only the course numbers (101, 205, 189) from the text above. How do you do that?

4.1 What does re.findall() do?

PyInteractive Python
import re text = """101 COM Computers 205 MAT Mathematics 189 ENG English""" print(text) print() regex_num = re.compile('\d+') print(regex_num.findall(text))

  

The special character '\d' matches any digit. Adding '+' requires at least one digit to be present.

There is also a '*' symbol. It requires zero or more digits, which makes the digit optional for a match. More on this later.

The findall method extracts all occurrences and returns them as a list.

4.2 re.search() vs re.match()

regex.search() looks for the pattern anywhere in the text. Unlike findall, it returns a match object with the start and end positions of the first match.

regex.match() also returns a match object. The difference? It only checks the beginning of the text.

PyInteractive Python
import re text2 = """COM Computers 205 MAT Mathematics 189""" regex_num = re.compile('\d+') s = regex_num.search(text2) print('Starting Position: ', s.start()) print('Ending Position: ', s.end()) print(text2[s.start():s.end()])

  

You can also get the matched text using the group() method:

PyInteractive Python
import re text2 = """COM Computers 205 MAT Mathematics 189""" regex_num = re.compile('\d+') s = regex_num.search(text2) print(s.group())

  

Now see what happens with match() when the text does not start with a digit:

PyInteractive Python
import re text2 = """COM Computers 205 MAT Mathematics 189""" regex_num = re.compile('\d+') m = regex_num.match(text2) print(m)

  

It returns None because there is no digit at the start of the string.

5. How to substitute one text with another using regex?

Use regex.sub() to replace text. Consider this version of the courses text with extra tabs after each course code:

PyInteractive Python
import re text = """101 COM \t Computers 205 MAT \t Mathematics 189 ENG \t English""" print(text)

  

You want to collapse all extra spaces into a single space and put everything on one line. Use regex.sub to replace '\s+' with a single space:

PyInteractive Python
import re text = """101 COM \t Computers 205 MAT \t Mathematics 189 ENG \t English""" regex = re.compile('\s+') print(regex.sub(' ', text)) print() print(re.sub('\s+', ' ', text))

  

What if you want to keep course entries on separate lines but remove extra spaces within each line? Use a negative lookahead (?!\n). It excludes newline characters from the match:

PyInteractive Python
import re text = """101 COM \t Computers 205 MAT \t Mathematics 189 ENG \t English""" regex = re.compile('((?!\n)\s+)') print(regex.sub(' ', text))

  

6. Regex groups

Regex groups let you extract matched parts as separate items. Suppose you want the course number, code, and name as individual pieces.

Without groups, you need three separate patterns:

PyInteractive Python
import re text = """101 COM Computers 205 MAT Mathematics 189 ENG English""" # Extract each part separately print(re.findall('[0-9]+', text)) print(re.findall('[A-Z]{3}', text)) print(re.findall('[A-Za-z]{4,}', text))

  

Here is what each pattern does:

[0-9]+ matches one or more digits (0 through 9). If you know the number has exactly 3 digits, use [0-9]{3} instead.

[A-Z]{3} matches exactly three uppercase letters A through Z.

[A-Za-z]{4,} matches four or more letters (upper or lower case). This assumes course names have at least 4 characters.

That took three separate calls. Regex groups offer a better way. Place each part inside parentheses () within a single pattern:

PyInteractive Python
import re text = """101 COM Computers 205 MAT Mathematics 189 ENG English""" course_pattern = '([0-9]+)\s*([A-Z]{3})\s*([A-Za-z]{4,})' print(re.findall(course_pattern, text))

  

Each set of parentheses creates a group. The result is a list of tuples with the matched groups.

7. What is greedy matching in regex?

By default, regex is greedy. It grabs as much text as possible while still matching the pattern.

Here is an example with HTML tags:

PyInteractive Python
import re text = "< body>Regex Greedy Matching Example < /body>" print(re.findall('<.*>', text))

  

Instead of stopping at the first >, regex grabbed the entire string. That is greedy behavior: “take it all.”

Lazy matching does the opposite: “take as little as possible.” Add ? after the pattern to make it lazy:

PyInteractive Python
import re text = "< body>Regex Greedy Matching Example < /body>" print(re.findall('<.*?>', text))

  

To get only the first match, use search instead of findall:

PyInteractive Python
import re text = "< body>Regex Greedy Matching Example < /body>" print(re.search('<.*?>', text).group())

  

8. Most common regular expression syntax and patterns

Now that you understand the re module, here are the most common wildcard patterns:

Basic Syntax
.             One character except new line
\.            A period. \ escapes a special character.
\d            One digit
\D            One non-digit
\w            One word character including digits
\W            One non-word character
\s            One whitespace
\S            One non-whitespace
\b            Word boundary
\n            Newline
\t            Tab

Modifiers
$             End of string
^             Start of string
ab|cd         Matches ab or de.
[ab-d]        One character of: a, b, c, d
[^ab-d]       One character except: a, b, c, d
()            Items within parenthesis are retrieved
(a(bc))       Items within the sub-parenthesis are retrieved

Repetitions
[ab]{2}       Exactly 2 continuous occurrences of a or b
[ab]{2,5}     2 to 5 continuous occurrences of a or b
[ab]{2,}      2 or more continuous occurrences of a or b
+             One or more
*             Zero or more
?             0 or 1

9. Regular Expressions Examples

9.1. Any character except for a new line

The dot . matches any single character except a newline:

PyInteractive Python
import re text = 'machinelearningplus.com' print(re.findall('.', text)) print(re.findall('...', text))

  

9.2. A period

Use \. to match a literal period:

PyInteractive Python
import re text = 'machinelearningplus.com' print(re.findall('\.', text)) print(re.findall('[^\.]', text))

  

9.3. Any digit

\d+ matches one or more digits:

PyInteractive Python
import re text = '01, Jan 2015' print(re.findall('\d+', text))

  

9.4. Anything but a digit

\D+ matches one or more non-digit characters:

PyInteractive Python
import re text = '01, Jan 2015' print(re.findall('\D+', text))

  

9.5. Any character, including digits

\w+ matches word characters (letters, digits, underscore):

PyInteractive Python
import re text = '01, Jan 2015' print(re.findall('\w+', text))

  

9.6. Anything but a character

\W+ matches non-word characters:

PyInteractive Python
import re text = '01, Jan 2015' print(re.findall('\W+', text))

  

9.7. Collection of characters

Square brackets [] match any character inside them:

PyInteractive Python
import re text = '01, Jan 2015' print(re.findall('[a-zA-Z]+', text))

  

9.8. Match something upto ‘n’ times

Use {n} to match exactly n repetitions:

PyInteractive Python
import re text = '01, Jan 2015' print(re.findall('\d{4}', text)) print(re.findall('\d{2,4}', text))

  

9.9. Match 1 or more occurrences

The + symbol matches one or more occurrences:

PyInteractive Python
import re print(re.findall(r'Co+l', 'So Cooool'))

  

9.10. Match any number of occurrences (0 or more times)

The * symbol matches zero or more occurrences:

PyInteractive Python
import re print(re.findall(r'Pi*lani', 'Pilani'))

  

9.11. Match exactly zero or one occurrence

The ? symbol matches zero or one occurrence:

PyInteractive Python
import re print(re.findall(r'colou?r', 'color'))

  

9.12. Match word boundaries

The \b boundary matches where one side is a word character and the other is whitespace. For example, \btoy matches ‘toy’ in ‘toy cat’ but not in ‘tolstoy’.

To match ‘toy’ in ‘tolstoy’, use toy\b. To match only the standalone word ‘toy’, place \b on both sides.

\B matches non-boundaries. So \Btoy\B matches ‘toy’ only when surrounded by word characters, as in ‘antoynet’.

PyInteractive Python
import re print(re.findall(r'\btoy\b', 'play toy broke toys'))

  

10. Practice Exercises

Try these exercises. Run the code blocks to check your answers.

Exercise 1: Extract the user id, domain name, and suffix from these email addresses.

PyInteractive Python
import re emails = """zuck26@facebook.com page33@google.com jeff42@amazon.com""" desired_output = [('zuck26', 'facebook', 'com'), ('page33', 'google', 'com'), ('jeff42', 'amazon', 'com')] print("Desired:", desired_output)

  

Solution: Use groups with () to capture each part:

PyInteractive Python
import re emails = """zuck26@facebook.com page33@google.com jeff42@amazon.com""" pattern = r'(\w+)@([A-Z0-9]+)\.([A-Z]{2,4})' result = re.findall(pattern, emails, flags=re.IGNORECASE) print(result)

  

Exercise 2: Retrieve all words starting with ‘b’ or ‘B’ from this text.

PyInteractive Python
import re text = """Betty bought a bit of butter, But the butter was so bitter, So she bought some better butter, To make the bitter butter better.""" result = re.findall(r'\bB\w+', text, flags=re.IGNORECASE) print(result)

  

The \b before ‘B’ requires a word boundary on the left, so the word must start with ‘B’. The re.IGNORECASE flag makes it case insensitive.

Exercise 3: Split this irregular sentence into clean words.

PyInteractive Python
import re sentence = """A, very very; irregular_sentence""" result = " ".join(re.split('[;,\s_]+', sentence)) print(result)

  

Exercise 4: Clean a tweet by removing URLs, hashtags, mentions, punctuation, RT, and CC.

PyInteractive Python
import re tweet = 'Good advice! RT @TheNextWeb: What I would do differently if I was learning to code today http://t.co/lbwej0pxOd cc: @garybernhardt #rstats' def clean_tweet(tweet): tweet = re.sub('http\S+\s*', '', tweet) # remove URLs tweet = re.sub('RT|cc', '', tweet) # remove RT and cc tweet = re.sub('#\S+', '', tweet) # remove hashtags tweet = re.sub('@\S+', '', tweet) # remove mentions tweet = re.sub('[%s]' % re.escape('!"#$%&\'()*+,-./:;<=>?@[\\]^_`{|}~'), '', tweet) # remove punctuations tweet = re.sub('\s+', ' ', tweet) # remove extra whitespace return tweet print(clean_tweet(tweet))

  

Exercise 5: Extract all text between HTML tags. This exercise requires the requests library, so here is the pattern to study:

import re
import requests

r = requests.get("https://raw.githubusercontent.com/selva86/datasets/master/sample.html")
print(re.findall('<.*?>(.*)', r.text))

The pattern <.*?>(.*)</.*?> captures everything between opening and closing tags.

11. Conclusion

This tutorial covered regular expressions in Python from the ground up. You learned how to compile patterns, split strings, find matches, substitute text, use groups, and control greedy vs. lazy matching.

Use the reference table and practice exercises to build your regex skills. Keep this guide handy for your next text mining project.