# Introduction
Why use Python? For many people it comes down to “just because,” but it really shouldn't. Python is a powerful, general-purpose programming language with a simple syntax highlighted by Pythonic approaches to logic and data management, which has proven to be the leading language for data science, machine learning and AI. precisely because of these reasons. Python is easy to pick up, but you can spend years working to improve your skills and understand the core mechanisms of the language, working to transition from a beginner to an expert who can write functional, maintainable systems.
With this in mind, today we're going to explore five basic concepts that every Python developer should have in their toolkit.
# 1. List Comprehension and Expression Generator
Python is famous for its readability. List comprehensions allow you to replace clunky loops with a single line of code. However, the real pro move here is knowing when to use the generator expression instead of storing it in memory.
// The Clunky Way (For the Loop)
Let's start with the less efficient, non-Pythonic “clunky” way of doing things:
numbers = range(1000000)
squared_list = []
for n in numbers:
if n % 2 == 0:
squared_list.append(n ** 2)// Pythonic Method (List of Comprehension)
Now let's look at the Pythonic way to solve the same task:
# Concise and faster execution
squared_list = [n ** 2 for n in numbers if n % 2 == 0]
# The "Must-Know" Twist: Generator Expressions
# If you only need to iterate once and don't need the whole list in memory:
squared_gen = (n ** 2 for n in numbers if n % 2 == 0)Output:
List size: 4,167,352 bytes
Generator size: 200 bytesHere's why this is important, without people telling you “that's how it's done in Python”: List comprehensions are very fast .append(). Generated expressions (using parentheses) are “lazy” — they generate objects once at a time, allowing you to process large data sets without overwhelming your program's memory.
Let's see how to use the generator, one call at a time, using the generator expression:
numbers = range(1000000)
squared_gen = (n ** 2 for n in numbers if n % 2 == 0)
# Values are computed only when requested, not all at once
print(next(squared_gen))
print(next(squared_gen))
print(next(squared_gen))Output:
# 2. The decorators
Decorators are a way to change the behavior of a function or class without permanently changing its source code. Think of them as wrappers around other functions.
// The Clunky Way
If you wanted to record how long several different tasks took to run, you could manually add a time code to every single task.
import time
def process_data():
start = time.time()
# ... function logic ...
end = time.time()
print(f"process_data took {end - start:.4f}s")
def train_model():
start = time.time()
# ... function logic ...
end = time.time()
print(f"train_model took {end - start:.4f}s")
def generate_report():
start = time.time()
# ... function logic ...
end = time.time()
print(f"generate_report took {end - start:.4f}s")Note that the repetition makes the problem obvious: the same four rows are doubled throughout the work. Let's see how a decorator can fix this.
// The Pythonic way
Here is a more Pythonic approach to this task.
import time
from functools import wraps
def timer_decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
start = time.time()
result = func(*args, **kwargs)
end = time.time()
print(f"{func.__name__} took {end - start:.4f}s")
return result
return wrapper
@timer_decorator
def heavy_computation():
return sum(range(10**7))
heavy_computation()Output:
heavy_computation took 0.0941sSee how timer_decorator() “wrapping” i heavy_computation() work, and if the latter is called upon, used, and reaps the benefits, of the former.
Decorators improve “don't do it again (DRY) principle. They are important for logging, authentication, and temporary storage in production facilities.
# 3. Context Managers (with statements)
Managing resources such as files, database connections, or network sockets is a common source of bugs. If you forget to close a file, you leak memory or lock the file in other processes.
// The Clunky Way
Here we open the file, use it, and force close it when it is no longer needed.
f = open("data.txt", "w")
try:
f.write("Hello World")
finally:
# Easy to forget!
f.close()// The Pythonic way
A statement with a statement can help us in the above.
# File is automatically closed here, even if an error occurs
with open("data.txt", "w") as f:
f.write("Hello World")Not only is it much shorter, it's more logical and easy to follow too – and you get to forget easily close() free, as “setting” and “breaking” happen reliably. For database operations, this is useful when connecting to a SQL database or handling large input/output (IO)-bound operations.
# 4. Teaching *args again **kwargs
Sometimes you don't know how many arguments will be passed to a function. Python handles this elegantly using the “packing” operator. Even as a beginner who may not have hired them, you've undoubtedly seen these “packer” operators at some point.
// A Pythonic example
Here's the Pythonic way to handle it:
*args(non-keyword arguments): The “packing” operator that collects more positional arguments in a tuple. This is used when you don't know how many items will be passed to the function.**kwargs(keyword arguments): The “packing” operator that collects more invented arguments in the dictionary. This is used for optional settings or named parameters.
def make_profile(name, *tags, **metadata):
# name is the named argument
print(f"User: {name}")
# tags is a tuple
print(f"Tags: {tags}")
# metadata is a dictionary
print(f"Details: {metadata}")
make_profile("Alice", "DataScientist", "Pythonist", location="NY", seniority="Senior")Output:
User: Alice
Tags: ('DataScientist', 'Pythonist')
Details: {'location': 'NY', 'seniority': 'Senior'}This is the secret behind dynamic libraries like Scikit-Learn or Matplotlib. It allows you to pass an arbitrary number of settings into a function, making your code incredibly adaptable to changing requirements.
# 5. Paths of Dunder (Paths of Magic)
“Dunder” stands for double underscore (eg __init__). Special formal methods (but often called magic methods), these methods allow your custom objects to mimic Python's built-in behavior.
// The Pythonic way
Let's see how we can use magic methods to add automatic behavior to our classes.
class Dataset:
def __init__(self, data):
self.data = data
def __len__(self):
return len(self.data)
def __str__(self):
return f"Dataset with {len(self.data)} items"
# Create a dataset instance
my_data = Dataset([1, 2, 3])
# Calls __len__
print(len(my_data))
# Calss __str__
print(my_data)Output:
By using the built-in __len__ again __str__ dunders, our custom class gets useful functionality for free.
Dunder methods are the core of the Python object protocol. By using methods like these __getitem__ or __call__you can make your classes behave like lists, dictionaries, or even functions, resulting in more intuitive APIs.
# Wrapping up
Mastering these five concepts marks the transition from writing scripts to building software. Using list comprehensions for speed, clean comprehension decorators, safe context managers, *args/**kwargs with flexibility, and dunder methods of object power, you lay the foundation on which you can build additional Python technology.
Matthew Mayo (@mattmayo13) has a master's degree in computer science and a diploma in data mining. As managing editor of KDnuggets & Statology, and contributing editor to Machine Learning Mastery, Matthew aims to make complex data science concepts accessible. His professional interests include natural language processing, language models, machine learning algorithms, and exploring emerging AI. He is driven by a mission to democratize knowledge in the data science community. Matthew has been coding since he was 6 years old.
